2026-10-30 19:51:30,844 [ 419436 ] INFO : ClickHouse root is not set. Will use /home/ubuntu/_work/ClickHouse/ClickHouse (runner:42, check_args_and_update_paths) 2026-10-30 19:51:30,844 [ 419436 ] INFO : Cases dir is not set. Will use /home/ubuntu/_work/ClickHouse/ClickHouse/tests/integration (runner:86, check_args_and_update_paths) 2026-10-30 19:51:30,844 [ 419436 ] INFO : utils dir is not set. Will use /home/ubuntu/_work/ClickHouse/ClickHouse/utils (runner:97, check_args_and_update_paths) 2026-10-30 19:51:30,844 [ 419436 ] INFO : base_configs_dir: /home/ubuntu/_work/ClickHouse/ClickHouse/programs/server, binary: /home/ubuntu/_work/_temp/test/build/clickhouse, cases_dir: /home/ubuntu/_work/ClickHouse/ClickHouse/tests/integration (runner:99, check_args_and_update_paths) clickhouse_integration_tests_volume Running pytest container as: 'docker run --rm --name clickhouse_integration_tests_96vjt7 --privileged --dns-search='.' --volume=/home/ubuntu/_work/_temp/test/build/clickhouse-odbc-bridge:/clickhouse-odbc-bridge --volume=/home/ubuntu/_work/_temp/test/build/clickhouse:/clickhouse --volume=/home/ubuntu/_work/_temp/test/build/clickhouse-library-bridge:/clickhouse-library-bridge --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/programs/server:/clickhouse-config --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/tests/integration:/ClickHouse/tests/integration --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/utils/backupview:/ClickHouse/utils/backupview --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/utils/grpc-client/pb2:/ClickHouse/utils/grpc-client/pb2 --volume=/run:/run/host:ro --volume=clickhouse_integration_tests_volume:/var/lib/docker -e DOCKER_DOTNET_CLIENT_TAG=11de0b29a15d -e DOCKER_HELPER_TAG=2cffe1eae894 -e DOCKER_BASE_TAG=b74d853ebe76 -e DOCKER_KERBEROS_KDC_TAG=9391ecdee8d7 -e DOCKER_MYSQL_GOLANG_CLIENT_TAG=9bec2a638e6e -e DOCKER_MYSQL_JAVA_CLIENT_TAG=766bff31cfe4 -e DOCKER_MYSQL_JS_CLIENT_TAG=41ba7c2ec2a1 -e DOCKER_MYSQL_PHP_CLIENT_TAG=88be89c1e3b6 -e DOCKER_NGINX_DAV_TAG=b55ac9cd7519 -e DOCKER_POSTGRESQL_JAVA_CLIENT_TAG=a4eff5c7f4d6 -e DOCKER_PYTHON_BOTTLE_TAG=caad4729259e -e DOCKER_CLIENT_TIMEOUT=300 -e COMPOSE_HTTP_TIMEOUT=600 -e PYTHONUNBUFFERED=1 -e PYTEST_ADDOPTS="--dist=loadfile -n 10 -rfEps --run-id=0 --color=no --durations=0 'test_storage_s3_queue/test.py::test_direct_select_file[unordered]' 'test_storage_s3_queue/test.py::test_direct_select_multiple_files[ordered]' 'test_storage_s3_queue/test.py::test_direct_select_multiple_files[unordered]' test_storage_s3_queue/test.py::test_drop_table test_storage_s3_queue/test.py::test_exception_during_insert 'test_storage_s3_queue/test.py::test_failed_retry[AzureQueue-ordered]' 'test_storage_s3_queue/test.py::test_failed_retry[AzureQueue-unordered]' 'test_storage_s3_queue/test.py::test_failed_retry[S3Queue-ordered]' 'test_storage_s3_queue/test.py::test_failed_retry[S3Queue-unordered]' test_storage_s3_queue/test.py::test_max_set_age test_storage_s3_queue/test.py::test_max_set_size test_storage_s3_queue/test.py::test_multiple_tables_meta_mismatch 'test_storage_s3_queue/test.py::test_multiple_tables_streaming_sync[unordered]' 'test_storage_s3_queue/test.py::test_multiple_tables_streaming_sync_distributed[ordered]' 'test_storage_s3_queue/test.py::test_multiple_tables_streaming_sync_distributed[unordered]' 'test_storage_s3_queue/test.py::test_processed_file_setting[1]' 'test_storage_s3_queue/test.py::test_processed_file_setting[5]' 'test_storage_s3_queue/test.py::test_processed_file_setting_distributed[1]' 'test_storage_s3_queue/test.py::test_processed_file_setting_distributed[5]' 'test_storage_s3_queue/test.py::test_processing_threads[ordered]' 'test_storage_s3_queue/test.py::test_processing_threads[unordered]' test_storage_s3_queue/test.py::test_s3_client_reused test_storage_s3_queue/test.py::test_settings_check 'test_storage_s3_queue/test.py::test_shards[ordered-1]' 'test_storage_s3_queue/test.py::test_shards[ordered-8]' 'test_storage_s3_queue/test.py::test_shards[unordered-1]' 'test_storage_s3_queue/test.py::test_shards[unordered-8]' 'test_storage_s3_queue/test.py::test_shards_distributed[ordered-1]' 'test_storage_s3_queue/test.py::test_shards_distributed[ordered-8]' 'test_storage_s3_queue/test.py::test_shards_distributed[unordered-1]' 'test_storage_s3_queue/test.py::test_shards_distributed[unordered-8]' 'test_storage_s3_queue/test.py::test_streaming_to_many_views[ordered]' 'test_storage_s3_queue/test.py::test_streaming_to_many_views[unordered]' 'test_storage_s3_queue/test.py::test_streaming_to_view_[ordered]' 'test_storage_s3_queue/test.py::test_streaming_to_view_[unordered]' test_storage_s3_queue/test.py::test_upgrade test_storage_s3_queue/test.py::test_upgrade_2 test_storage_url_with_proxy/test.py::test_s3_with_proxy_list test_system_ddl_worker_queue/test.py::test_distributed_ddl_queue test_system_ddl_worker_queue/test.py::test_distributed_ddl_rubbish test_system_flush_logs/test.py::test_system_logs_exists test_system_flush_logs/test.py::test_system_logs_non_empty_queue test_system_flush_logs/test.py::test_system_suspend test_system_queries/test.py::test_DROP_DNS_CACHE test_system_queries/test.py::test_RELOAD_CONFIG_AND_MACROS test_system_queries/test.py::test_SYSTEM_RELOAD_DICTIONARY test_system_queries/test.py::test_system_flush_logs 'test_throttling/test.py::test_backup_throttling[no_local_throttling]' 'test_throttling/test.py::test_backup_throttling[no_local_to_remote_throttling]' 'test_throttling/test.py::test_backup_throttling[no_remote_to_local_throttling]' 'test_throttling/test.py::test_backup_throttling[no_remote_to_remote_throttling]' 'test_throttling/test.py::test_backup_throttling[server_local_throttling]' 'test_throttling/test.py::test_backup_throttling[server_local_to_remote_throttling]' 'test_throttling/test.py::test_backup_throttling[server_remote_to_local_throttling]' 'test_throttling/test.py::test_backup_throttling[server_remote_to_remote_throttling]' 'test_throttling/test.py::test_backup_throttling[user_local_throttling]' 'test_throttling/test.py::test_backup_throttling[user_local_to_remote_throttling]' 'test_throttling/test.py::test_backup_throttling[user_remote_to_local_throttling]' 'test_throttling/test.py::test_backup_throttling[user_remote_to_remote_throttling]' test_throttling/test.py::test_backup_throttling_override test_throttling/test.py::test_max_merges_bandwidth_for_server test_throttling/test.py::test_max_mutations_bandwidth_for_server 'test_throttling/test.py::test_read_throttling[no_local_throttling]' 'test_throttling/test.py::test_read_throttling[no_remote_throttling]' 'test_throttling/test.py::test_read_throttling[server_local_throttling]' 'test_throttling/test.py::test_read_throttling[server_remote_throttling]' 'test_throttling/test.py::test_read_throttling[user_local_throttling]' 'test_throttling/test.py::test_read_throttling[user_remote_throttling]' 'test_throttling/test.py::test_write_throttling[local_server_throttling]' 'test_throttling/test.py::test_write_throttling[local_user_throttling]' 'test_throttling/test.py::test_write_throttling[no_local_throttling]' 'test_throttling/test.py::test_write_throttling[no_remote_throttling]' 'test_throttling/test.py::test_write_throttling[server_remote_throttling]' 'test_throttling/test.py::test_write_throttling[user_remote_throttling]' test_transactions/test.py::test_rollback_unfinished_on_restart1 test_transactions/test.py::test_rollback_unfinished_on_restart2 test_truncate_database/test_distributed.py::test_truncate_database_distributed test_undrop_query/test.py::test_undrop_drop_and_undrop_loop test_user_defined_object_persistence/test.py::test_persistence test_user_zero_database_access/test_user_zero_database_access.py::test_user_zero_database_access test_version_update_after_mutation/test.py::test_mutate_and_upgrade test_version_update_after_mutation/test.py::test_upgrade_while_mutation test_zookeeper_config/test_secure.py::test_secure_connection -vvv" altinityinfra/integration-tests-runner:cf95077f4609 '. Start tests ============================= test session starts ============================== platform linux -- Python 3.10.12, pytest-7.4.4, pluggy-1.5.0 -- /usr/bin/python3 cachedir: .pytest_cache rootdir: /ClickHouse/tests/integration configfile: pytest.ini plugins: random-0.2, reportlog-0.4.0, xdist-3.5.0, repeat-0.9.4, order-1.0.1, timeout-2.2.0 timeout: 900.0s timeout method: signal timeout func_only: False created: 10/10 workers 10 workers [83 items] scheduling tests via LoadFileScheduling test_transactions/test.py::test_rollback_unfinished_on_restart1 test_system_ddl_worker_queue/test.py::test_distributed_ddl_queue test_truncate_database/test_distributed.py::test_truncate_database_distributed test_storage_url_with_proxy/test.py::test_s3_with_proxy_list test_system_flush_logs/test.py::test_system_logs_exists test_undrop_query/test.py::test_undrop_drop_and_undrop_loop test_storage_s3_queue/test.py::test_direct_select_file[unordered] test_system_queries/test.py::test_DROP_DNS_CACHE test_version_update_after_mutation/test.py::test_mutate_and_upgrade test_throttling/test.py::test_backup_throttling[no_local_throttling] [gw4] [ 1%] PASSED test_system_flush_logs/test.py::test_system_logs_exists test_system_flush_logs/test.py::test_system_logs_non_empty_queue [gw4] [ 2%] PASSED test_system_flush_logs/test.py::test_system_logs_non_empty_queue test_system_flush_logs/test.py::test_system_suspend [gw3] [ 3%] PASSED test_system_queries/test.py::test_DROP_DNS_CACHE test_system_queries/test.py::test_RELOAD_CONFIG_AND_MACROS [gw4] [ 4%] PASSED test_system_flush_logs/test.py::test_system_suspend [gw3] [ 6%] PASSED test_system_queries/test.py::test_RELOAD_CONFIG_AND_MACROS test_system_queries/test.py::test_SYSTEM_RELOAD_DICTIONARY [gw3] [ 7%] PASSED test_system_queries/test.py::test_SYSTEM_RELOAD_DICTIONARY test_system_queries/test.py::test_system_flush_logs [gw7] [ 8%] PASSED test_storage_url_with_proxy/test.py::test_s3_with_proxy_list [gw2] [ 9%] PASSED test_throttling/test.py::test_backup_throttling[no_local_throttling] [gw3] [ 10%] PASSED test_system_queries/test.py::test_system_flush_logs test_throttling/test.py::test_backup_throttling[no_local_to_remote_throttling] [gw8] [ 12%] PASSED test_truncate_database/test_distributed.py::test_truncate_database_distributed [gw2] [ 13%] PASSED test_throttling/test.py::test_backup_throttling[no_local_to_remote_throttling] [gw5] [ 14%] PASSED test_transactions/test.py::test_rollback_unfinished_on_restart1 test_transactions/test.py::test_rollback_unfinished_on_restart2 [gw1] [ 15%] PASSED test_system_ddl_worker_queue/test.py::test_distributed_ddl_queue test_system_ddl_worker_queue/test.py::test_distributed_ddl_rubbish test_throttling/test.py::test_backup_throttling[no_remote_to_local_throttling] [gw2] [ 16%] PASSED test_throttling/test.py::test_backup_throttling[no_remote_to_local_throttling] [gw9] [ 18%] PASSED test_undrop_query/test.py::test_undrop_drop_and_undrop_loop [gw5] [ 19%] PASSED test_transactions/test.py::test_rollback_unfinished_on_restart2 [gw1] [ 20%] PASSED test_system_ddl_worker_queue/test.py::test_distributed_ddl_rubbish test_throttling/test.py::test_backup_throttling[no_remote_to_remote_throttling] [gw2] [ 21%] PASSED test_throttling/test.py::test_backup_throttling[no_remote_to_remote_throttling] test_user_zero_database_access/test_user_zero_database_access.py::test_user_zero_database_access [gw6] [ 22%] PASSED test_version_update_after_mutation/test.py::test_mutate_and_upgrade test_version_update_after_mutation/test.py::test_upgrade_while_mutation test_throttling/test.py::test_backup_throttling[server_local_throttling] [gw6] [ 24%] PASSED test_version_update_after_mutation/test.py::test_upgrade_while_mutation test_user_defined_object_persistence/test.py::test_persistence [gw5] [ 25%] PASSED test_user_zero_database_access/test_user_zero_database_access.py::test_user_zero_database_access test_zookeeper_config/test_secure.py::test_secure_connection [gw2] [ 26%] PASSED test_throttling/test.py::test_backup_throttling[server_local_throttling] test_throttling/test.py::test_backup_throttling[server_local_to_remote_throttling] [gw1] [ 27%] PASSED test_user_defined_object_persistence/test.py::test_persistence [gw2] [ 28%] PASSED test_throttling/test.py::test_backup_throttling[server_local_to_remote_throttling] test_throttling/test.py::test_backup_throttling[server_remote_to_local_throttling] [gw2] [ 30%] PASSED test_throttling/test.py::test_backup_throttling[server_remote_to_local_throttling] test_throttling/test.py::test_backup_throttling[server_remote_to_remote_throttling] [gw2] [ 31%] PASSED test_throttling/test.py::test_backup_throttling[server_remote_to_remote_throttling] [gw6] [ 32%] PASSED test_zookeeper_config/test_secure.py::test_secure_connection test_throttling/test.py::test_backup_throttling[user_local_throttling] [gw2] [ 33%] PASSED test_throttling/test.py::test_backup_throttling[user_local_throttling] test_throttling/test.py::test_backup_throttling[user_local_to_remote_throttling] [gw2] [ 34%] PASSED test_throttling/test.py::test_backup_throttling[user_local_to_remote_throttling] test_throttling/test.py::test_backup_throttling[user_remote_to_local_throttling] [gw2] [ 36%] PASSED test_throttling/test.py::test_backup_throttling[user_remote_to_local_throttling] test_throttling/test.py::test_backup_throttling[user_remote_to_remote_throttling] [gw2] [ 37%] PASSED test_throttling/test.py::test_backup_throttling[user_remote_to_remote_throttling] test_throttling/test.py::test_backup_throttling_override [gw2] [ 38%] PASSED test_throttling/test.py::test_backup_throttling_override test_throttling/test.py::test_max_merges_bandwidth_for_server [gw2] [ 39%] PASSED test_throttling/test.py::test_max_merges_bandwidth_for_server test_throttling/test.py::test_max_mutations_bandwidth_for_server [gw2] [ 40%] PASSED test_throttling/test.py::test_max_mutations_bandwidth_for_server test_throttling/test.py::test_read_throttling[no_local_throttling] [gw2] [ 42%] PASSED test_throttling/test.py::test_read_throttling[no_local_throttling] test_throttling/test.py::test_read_throttling[no_remote_throttling] [gw2] [ 43%] PASSED test_throttling/test.py::test_read_throttling[no_remote_throttling] test_throttling/test.py::test_read_throttling[server_local_throttling] [gw2] [ 44%] PASSED test_throttling/test.py::test_read_throttling[server_local_throttling] test_throttling/test.py::test_read_throttling[server_remote_throttling] [gw2] [ 45%] PASSED test_throttling/test.py::test_read_throttling[server_remote_throttling] test_throttling/test.py::test_read_throttling[user_local_throttling] [gw2] [ 46%] PASSED test_throttling/test.py::test_read_throttling[user_local_throttling] test_throttling/test.py::test_read_throttling[user_remote_throttling] [gw2] [ 48%] PASSED test_throttling/test.py::test_read_throttling[user_remote_throttling] test_throttling/test.py::test_write_throttling[local_server_throttling] [gw2] [ 49%] PASSED test_throttling/test.py::test_write_throttling[local_server_throttling] test_throttling/test.py::test_write_throttling[local_user_throttling] [gw2] [ 50%] PASSED test_throttling/test.py::test_write_throttling[local_user_throttling] [gw0] [ 51%] ERROR test_storage_s3_queue/test.py::test_direct_select_file[unordered] test_storage_s3_queue/test.py::test_direct_select_multiple_files[ordered] [gw0] [ 53%] ERROR test_storage_s3_queue/test.py::test_direct_select_multiple_files[ordered] test_storage_s3_queue/test.py::test_direct_select_multiple_files[unordered] [gw0] [ 54%] ERROR test_storage_s3_queue/test.py::test_direct_select_multiple_files[unordered] test_storage_s3_queue/test.py::test_drop_table [gw0] [ 55%] ERROR test_storage_s3_queue/test.py::test_drop_table test_storage_s3_queue/test.py::test_exception_during_insert [gw0] [ 56%] ERROR test_storage_s3_queue/test.py::test_exception_during_insert test_storage_s3_queue/test.py::test_failed_retry[AzureQueue-ordered] [gw0] [ 57%] ERROR test_storage_s3_queue/test.py::test_failed_retry[AzureQueue-ordered] test_storage_s3_queue/test.py::test_failed_retry[AzureQueue-unordered] [gw0] [ 59%] ERROR test_storage_s3_queue/test.py::test_failed_retry[AzureQueue-unordered] test_storage_s3_queue/test.py::test_failed_retry[S3Queue-ordered] [gw0] [ 60%] ERROR test_storage_s3_queue/test.py::test_failed_retry[S3Queue-ordered] test_storage_s3_queue/test.py::test_failed_retry[S3Queue-unordered] [gw0] [ 61%] ERROR test_storage_s3_queue/test.py::test_failed_retry[S3Queue-unordered] test_storage_s3_queue/test.py::test_max_set_age [gw0] [ 62%] ERROR test_storage_s3_queue/test.py::test_max_set_age test_storage_s3_queue/test.py::test_max_set_size [gw0] [ 63%] ERROR test_storage_s3_queue/test.py::test_max_set_size test_storage_s3_queue/test.py::test_multiple_tables_meta_mismatch [gw0] [ 65%] ERROR test_storage_s3_queue/test.py::test_multiple_tables_meta_mismatch test_storage_s3_queue/test.py::test_multiple_tables_streaming_sync[unordered] [gw0] [ 66%] ERROR test_storage_s3_queue/test.py::test_multiple_tables_streaming_sync[unordered] test_storage_s3_queue/test.py::test_multiple_tables_streaming_sync_distributed[ordered] [gw0] [ 67%] ERROR test_storage_s3_queue/test.py::test_multiple_tables_streaming_sync_distributed[ordered] test_storage_s3_queue/test.py::test_multiple_tables_streaming_sync_distributed[unordered] [gw0] [ 68%] ERROR test_storage_s3_queue/test.py::test_multiple_tables_streaming_sync_distributed[unordered] test_storage_s3_queue/test.py::test_processed_file_setting[1] [gw0] [ 69%] ERROR test_storage_s3_queue/test.py::test_processed_file_setting[1] test_storage_s3_queue/test.py::test_processed_file_setting[5] [gw0] [ 71%] ERROR test_storage_s3_queue/test.py::test_processed_file_setting[5] test_storage_s3_queue/test.py::test_processed_file_setting_distributed[1] [gw0] [ 72%] ERROR test_storage_s3_queue/test.py::test_processed_file_setting_distributed[1] test_storage_s3_queue/test.py::test_processed_file_setting_distributed[5] [gw0] [ 73%] ERROR test_storage_s3_queue/test.py::test_processed_file_setting_distributed[5] test_storage_s3_queue/test.py::test_processing_threads[ordered] test_throttling/test.py::test_write_throttling[no_local_throttling] [gw0] [ 74%] ERROR test_storage_s3_queue/test.py::test_processing_threads[ordered] test_storage_s3_queue/test.py::test_processing_threads[unordered] [gw2] [ 75%] PASSED test_throttling/test.py::test_write_throttling[no_local_throttling] [gw0] [ 77%] ERROR test_storage_s3_queue/test.py::test_processing_threads[unordered] test_storage_s3_queue/test.py::test_s3_client_reused [gw0] [ 78%] ERROR test_storage_s3_queue/test.py::test_s3_client_reused test_storage_s3_queue/test.py::test_settings_check [gw0] [ 79%] ERROR test_storage_s3_queue/test.py::test_settings_check test_storage_s3_queue/test.py::test_shards[ordered-1] [gw0] [ 80%] ERROR test_storage_s3_queue/test.py::test_shards[ordered-1] test_storage_s3_queue/test.py::test_shards[ordered-8] [gw0] [ 81%] ERROR test_storage_s3_queue/test.py::test_shards[ordered-8] test_storage_s3_queue/test.py::test_shards[unordered-1] [gw0] [ 83%] ERROR test_storage_s3_queue/test.py::test_shards[unordered-1] test_storage_s3_queue/test.py::test_shards[unordered-8] [gw0] [ 84%] ERROR test_storage_s3_queue/test.py::test_shards[unordered-8] test_storage_s3_queue/test.py::test_shards_distributed[ordered-1] [gw0] [ 85%] ERROR test_storage_s3_queue/test.py::test_shards_distributed[ordered-1] test_storage_s3_queue/test.py::test_shards_distributed[ordered-8] [gw0] [ 86%] ERROR test_storage_s3_queue/test.py::test_shards_distributed[ordered-8] test_storage_s3_queue/test.py::test_shards_distributed[unordered-1] [gw0] [ 87%] ERROR test_storage_s3_queue/test.py::test_shards_distributed[unordered-1] test_storage_s3_queue/test.py::test_shards_distributed[unordered-8] [gw0] [ 89%] ERROR test_storage_s3_queue/test.py::test_shards_distributed[unordered-8] test_storage_s3_queue/test.py::test_streaming_to_many_views[ordered] [gw0] [ 90%] ERROR test_storage_s3_queue/test.py::test_streaming_to_many_views[ordered] test_storage_s3_queue/test.py::test_streaming_to_many_views[unordered] [gw0] [ 91%] ERROR test_storage_s3_queue/test.py::test_streaming_to_many_views[unordered] test_storage_s3_queue/test.py::test_streaming_to_view_[ordered] [gw0] [ 92%] ERROR test_storage_s3_queue/test.py::test_streaming_to_view_[ordered] test_storage_s3_queue/test.py::test_streaming_to_view_[unordered] [gw0] [ 93%] ERROR test_storage_s3_queue/test.py::test_streaming_to_view_[unordered] test_storage_s3_queue/test.py::test_upgrade [gw0] [ 95%] ERROR test_storage_s3_queue/test.py::test_upgrade test_storage_s3_queue/test.py::test_upgrade_2 [gw0] [ 96%] ERROR test_storage_s3_queue/test.py::test_upgrade_2 test_throttling/test.py::test_write_throttling[no_remote_throttling] [gw2] [ 97%] PASSED test_throttling/test.py::test_write_throttling[no_remote_throttling] test_throttling/test.py::test_write_throttling[server_remote_throttling] [gw2] [ 98%] PASSED test_throttling/test.py::test_write_throttling[server_remote_throttling] test_throttling/test.py::test_write_throttling[user_remote_throttling] [gw2] [100%] PASSED test_throttling/test.py::test_write_throttling[user_remote_throttling] ==================================== ERRORS ==================================== _____________ ERROR at setup of test_direct_select_file[unordered] _____________ [gw0] linux -- Python 3.10.12 /usr/bin/python3 @pytest.fixture(scope="module") def started_cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "instance", user_configs=["configs/users.xml"], with_minio=True, with_azurite=True, with_zookeeper=True, main_configs=[ "configs/zookeeper.xml", "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "instance2", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "old_instance", user_configs=["configs/users_old.xml"], with_zookeeper=True, # NOTE (vnemkov) Can't use altinity/clickhouse here since 23.8 doesn't hve S3Queue (and associated settings yet) image="clickhouse/clickhouse-server", tag="23.12", stay_alive=True, with_installed_binary=True, use_old_analyzer=True, ) cluster.add_instance( "instance_too_many_parts", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", "configs/merge_tree.xml", ], stay_alive=True, ) cluster.add_instance( "instance_24.5", with_zookeeper=True, image="clickhouse/clickhouse-server", tag="24.5", stay_alive=True, user_configs=[ "configs/users.xml", ], with_installed_binary=True, use_old_analyzer=True, ) logging.info("Starting cluster...") > cluster.start() test_storage_s3_queue/test.py:144: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:3119: in start self.wait_azurite_to_start() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = timeout = 180 def wait_azurite_to_start(self, timeout=180): from azure.storage.blob import BlobServiceClient connection_string = ( f"DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;" f"AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;" f"BlobEndpoint=http://127.0.0.1:{self.env_variables['AZURITE_PORT']}/devstoreaccount1;" ) time.sleep(1) start = time.time() while time.time() - start < timeout: try: blob_service_client = BlobServiceClient.from_connection_string( connection_string ) logging.debug(blob_service_client.get_account_information()) self.blob_service_client = blob_service_client return except Exception as ex: logging.debug("Can't connect to Azurite: %s", str(ex)) time.sleep(1) > raise Exception("Can't wait Azurite to start") E Exception: Can't wait Azurite to start helpers/cluster.py:2706: Exception ---------------------------- Captured stdout setup ----------------------------- Copy common default production configuration from /clickhouse-config. Files: config.xml, users.xml Copy common default production configuration from /clickhouse-config. Files: config.xml, users.xml Copy common default production configuration from /clickhouse-config. Files: config.xml, users.xml Copy common default production configuration from /clickhouse-config. Files: config.xml, users.xml Copy common default production configuration from /clickhouse-config. Files: config.xml, users.xml ---------------------------- Captured stderr setup ----------------------------- Traceback (most recent call last): File "/ClickHouse/tests/integration/helpers/cluster.py", line 3119, in start self.wait_azurite_to_start() File "/ClickHouse/tests/integration/helpers/cluster.py", line 2706, in wait_azurite_to_start raise Exception("Can't wait Azurite to start") Exception: Can't wait Azurite to start ------------------------------ Captured log setup ------------------------------ 2026-10-30 18:51:39 [ 408 ] DEBUG : Command:['docker ps | wc -l'] (cluster.py:113, run_and_check) 2026-10-30 18:51:40 [ 408 ] DEBUG : Stdout:1 (cluster.py:121, run_and_check) 2026-10-30 18:51:40 [ 408 ] DEBUG : No running containers (conftest.py:95, cleanup_environment) 2026-10-30 18:51:40 [ 408 ] DEBUG : Pruning Docker networks (conftest.py:97, cleanup_environment) 2026-10-30 18:51:40 [ 408 ] DEBUG : Command:['docker network prune --force'] (cluster.py:113, run_and_check) 2026-10-30 18:51:40 [ 408 ] DEBUG : Command:["sysctl net.ipv4.ip_local_port_range='55000 65535'"] (cluster.py:113, run_and_check) 2026-10-30 18:51:40 [ 408 ] DEBUG : Stdout:net.ipv4.ip_local_port_range = 55000 65535 (cluster.py:121, run_and_check) 2026-10-30 18:51:40 [ 408 ] DEBUG : ENV DOCKER_KERBEROS_KDC_TAG 9391ecdee8d7 (cluster.py:446, __init__) 2026-10-30 18:51:40 [ 408 ] DEBUG : ENV CLICKHOUSE_TESTS_SERVER_BIN_PATH /clickhouse (cluster.py:446, __init__) 2026-10-30 18:51:40 [ 408 ] DEBUG : ENV MSAN_OPTIONS abort_on_error=1 poison_in_dtor=1 (cluster.py:446, __init__) 2026-10-30 18:51:40 [ 408 ] DEBUG : ENV JAVA_TOOL_OPTIONS -Djdk.attach.allowAttachSelf=true (cluster.py:446, __init__) 2026-10-30 18:51:40 [ 408 ] DEBUG : ENV TSAN_OPTIONS halt_on_error=1 abort_on_error=1 history_size=7 memory_limit_mb=46080 second_deadlock_stack=1 (cluster.py:446, __init__) 2026-10-30 18:51:40 [ 408 ] DEBUG : ENV HOSTNAME 0b0f5ee01f52 (cluster.py:446, __init__) 2026-10-30 18:51:40 [ 408 ] DEBUG : ENV SHLVL 0 (cluster.py:446, __init__) 2026-10-30 18:51:40 [ 408 ] DEBUG : ENV HOME /root (cluster.py:446, __init__) 2026-10-30 18:51:40 [ 408 ] DEBUG : ENV OLDPWD / (cluster.py:446, __init__) 2026-10-30 18:51:40 [ 408 ] DEBUG : ENV DOCKER_HELPER_TAG 2cffe1eae894 (cluster.py:446, __init__) 2026-10-30 18:51:40 [ 408 ] DEBUG : ENV PYTHONUNBUFFERED 1 (cluster.py:446, __init__) 2026-10-30 18:51:40 [ 408 ] DEBUG : ENV DOCKER_PYTHON_BOTTLE_TAG caad4729259e (cluster.py:446, __init__) 2026-10-30 18:51:40 [ 408 ] DEBUG : ENV UBSAN_OPTIONS print_stacktrace=1 (cluster.py:446, __init__) 2026-10-30 18:51:40 [ 408 ] DEBUG : ENV PYTEST_ADDOPTS --dist=loadfile -n 10 -rfEps --run-id=0 --color=no --durations=0 'test_storage_s3_queue/test.py::test_direct_select_file[unordered]' 'test_storage_s3_queue/test.py::test_direct_select_multiple_files[ordered]' 'test_storage_s3_queue/test.py::test_direct_select_multiple_files[unordered]' test_storage_s3_queue/test.py::test_drop_table test_storage_s3_queue/test.py::test_exception_during_insert 'test_storage_s3_queue/test.py::test_failed_retry[AzureQueue-ordered]' 'test_storage_s3_queue/test.py::test_failed_retry[AzureQueue-unordered]' 'test_storage_s3_queue/test.py::test_failed_retry[S3Queue-ordered]' 'test_storage_s3_queue/test.py::test_failed_retry[S3Queue-unordered]' test_storage_s3_queue/test.py::test_max_set_age test_storage_s3_queue/test.py::test_max_set_size test_storage_s3_queue/test.py::test_multiple_tables_meta_mismatch 'test_storage_s3_queue/test.py::test_multiple_tables_streaming_sync[unordered]' 'test_storage_s3_queue/test.py::test_multiple_tables_streaming_sync_distributed[ordered]' 'test_storage_s3_queue/test.py::test_multiple_tables_streaming_sync_distributed[unordered]' 'test_storage_s3_queue/test.py::test_processed_file_setting[1]' 'test_storage_s3_queue/test.py::test_processed_file_setting[5]' 'test_storage_s3_queue/test.py::test_processed_file_setting_distributed[1]' 'test_storage_s3_queue/test.py::test_processed_file_setting_distributed[5]' 'test_storage_s3_queue/test.py::test_processing_threads[ordered]' 'test_storage_s3_queue/test.py::test_processing_threads[unordered]' test_storage_s3_queue/test.py::test_s3_client_reused test_storage_s3_queue/test.py::test_settings_check 'test_storage_s3_queue/test.py::test_shards[ordered-1]' 'test_storage_s3_queue/test.py::test_shards[ordered-8]' 'test_storage_s3_queue/test.py::test_shards[unordered-1]' 'test_storage_s3_queue/test.py::test_shards[unordered-8]' 'test_storage_s3_queue/test.py::test_shards_distributed[ordered-1]' 'test_storage_s3_queue/test.py::test_shards_distributed[ordered-8]' 'test_storage_s3_queue/test.py::test_shards_distributed[unordered-1]' 'test_storage_s3_queue/test.py::test_shards_distributed[unordered-8]' 'test_storage_s3_queue/test.py::test_streaming_to_many_views[ordered]' 'test_storage_s3_queue/test.py::test_streaming_to_many_views[unordered]' 'test_storage_s3_queue/test.py::test_streaming_to_view_[ordered]' 'test_storage_s3_queue/test.py::test_streaming_to_view_[unordered]' test_storage_s3_queue/test.py::test_upgrade test_storage_s3_queue/test.py::test_upgrade_2 test_storage_url_with_proxy/test.py::test_s3_with_proxy_list test_system_ddl_worker_queue/test.py::test_distributed_ddl_queue test_system_ddl_worker_queue/test.py::test_distributed_ddl_rubbish test_system_flush_logs/test.py::test_system_logs_exists test_system_flush_logs/test.py::test_system_logs_non_empty_queue test_system_flush_logs/test.py::test_system_suspend test_system_queries/test.py::test_DROP_DNS_CACHE test_system_queries/test.py::test_RELOAD_CONFIG_AND_MACROS test_system_queries/test.py::test_SYSTEM_RELOAD_DICTIONARY test_system_queries/test.py::test_system_flush_logs 'test_throttling/test.py::test_backup_throttling[no_local_throttling]' 'test_throttling/test.py::test_backup_throttling[no_local_to_remote_throttling]' 'test_throttling/test.py::test_backup_throttling[no_remote_to_local_throttling]' 'test_throttling/test.py::test_backup_throttling[no_remote_to_remote_throttling]' 'test_throttling/test.py::test_backup_throttling[server_local_throttling]' 'test_throttling/test.py::test_backup_throttling[server_local_to_remote_throttling]' 'test_throttling/test.py::test_backup_throttling[server_remote_to_local_throttling]' 'test_throttling/test.py::test_backup_throttling[server_remote_to_remote_throttling]' 'test_throttling/test.py::test_backup_throttling[user_local_throttling]' 'test_throttling/test.py::test_backup_throttling[user_local_to_remote_throttling]' 'test_throttling/test.py::test_backup_throttling[user_remote_to_local_throttling]' 'test_throttling/test.py::test_backup_throttling[user_remote_to_remote_throttling]' test_throttling/test.py::test_backup_throttling_override test_throttling/test.py::test_max_merges_bandwidth_for_server test_throttling/test.py::test_max_mutations_bandwidth_for_server 'test_throttling/test.py::test_read_throttling[no_local_throttling]' 'test_throttling/test.py::test_read_throttling[no_remote_throttling]' 'test_throttling/test.py::test_read_throttling[server_local_throttling]' 'test_throttling/test.py::test_read_throttling[server_remote_throttling]' 'test_throttling/test.py::test_read_throttling[user_local_throttling]' 'test_throttling/test.py::test_read_throttling[user_remote_throttling]' 'test_throttling/test.py::test_write_throttling[local_server_throttling]' 'test_throttling/test.py::test_write_throttling[local_user_throttling]' 'test_throttling/test.py::test_write_throttling[no_local_throttling]' 'test_throttling/test.py::test_write_throttling[no_remote_throttling]' 'test_throttling/test.py::test_write_throttling[server_remote_throttling]' 'test_throttling/test.py::test_write_throttling[user_remote_throttling]' test_transactions/test.py::test_rollback_unfinished_on_restart1 test_transactions/test.py::test_rollback_unfinished_on_restart2 test_truncate_database/test_distributed.py::test_truncate_database_distributed test_undrop_query/test.py::test_undrop_drop_and_undrop_loop test_user_defined_object_persistence/test.py::test_persistence test_user_zero_database_access/test_user_zero_database_access.py::test_user_zero_database_access test_version_update_after_mutation/test.py::test_mutate_and_upgrade test_version_update_after_mutation/test.py::test_upgrade_while_mutation test_zookeeper_config/test_secure.py::test_secure_connection -vvv (cluster.py:446, __init__) 2026-10-30 18:51:40 [ 408 ] DEBUG : ENV CLICKHOUSE_LIBRARY_BRIDGE_BINARY_PATH /clickhouse-library-bridge (cluster.py:446, __init__) 2026-10-30 18:51:40 [ 408 ] DEBUG : ENV COMPOSE_HTTP_TIMEOUT 600 (cluster.py:446, __init__) 2026-10-30 18:51:40 [ 408 ] DEBUG : ENV DOCKER_MYSQL_PHP_CLIENT_TAG 88be89c1e3b6 (cluster.py:446, __init__) 2026-10-30 18:51:40 [ 408 ] DEBUG : ENV DOCKER_DOTNET_CLIENT_TAG 11de0b29a15d (cluster.py:446, __init__) 2026-10-30 18:51:40 [ 408 ] DEBUG : ENV CLICKHOUSE_TESTS_CLIENT_BIN_PATH /clickhouse (cluster.py:446, __init__) 2026-10-30 18:51:40 [ 408 ] DEBUG : ENV DOCKER_MYSQL_JS_CLIENT_TAG 41ba7c2ec2a1 (cluster.py:446, __init__) 2026-10-30 18:51:40 [ 408 ] DEBUG : ENV PATH /spark-3.3.2-bin-hadoop3/bin:/opt/gdb/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin (cluster.py:446, __init__) 2026-10-30 18:51:40 [ 408 ] DEBUG : ENV DOCKER_KERBERIZED_HADOOP_TAG latest (cluster.py:446, __init__) 2026-10-30 18:51:40 [ 408 ] DEBUG : ENV DOCKER_CHANNEL stable (cluster.py:446, __init__) 2026-10-30 18:51:40 [ 408 ] DEBUG : ENV DOCKER_CLIENT_TIMEOUT 300 (cluster.py:446, __init__) 2026-10-30 18:51:40 [ 408 ] DEBUG : ENV DOCKER_POSTGRESQL_JAVA_CLIENT_TAG a4eff5c7f4d6 (cluster.py:446, __init__) 2026-10-30 18:51:40 [ 408 ] DEBUG : ENV DOCKER_NGINX_DAV_TAG b55ac9cd7519 (cluster.py:446, __init__) 2026-10-30 18:51:40 [ 408 ] DEBUG : ENV DOCKER_MYSQL_GOLANG_CLIENT_TAG 9bec2a638e6e (cluster.py:446, __init__) 2026-10-30 18:51:40 [ 408 ] DEBUG : ENV PWD /ClickHouse/tests/integration (cluster.py:446, __init__) 2026-10-30 18:51:40 [ 408 ] DEBUG : ENV DOCKER_MYSQL_JAVA_CLIENT_TAG 766bff31cfe4 (cluster.py:446, __init__) 2026-10-30 18:51:40 [ 408 ] DEBUG : ENV CLICKHOUSE_ODBC_BRIDGE_BINARY_PATH /clickhouse-odbc-bridge (cluster.py:446, __init__) 2026-10-30 18:51:40 [ 408 ] DEBUG : ENV CLICKHOUSE_TESTS_BASE_CONFIG_DIR /clickhouse-config (cluster.py:446, __init__) 2026-10-30 18:51:40 [ 408 ] DEBUG : ENV TZ Etc/UTC (cluster.py:446, __init__) 2026-10-30 18:51:40 [ 408 ] DEBUG : ENV JAVA_PATH /usr/lib/jvm/java-11-openjdk-arm64/bin/java (cluster.py:446, __init__) 2026-10-30 18:51:40 [ 408 ] DEBUG : ENV DOCKER_BASE_TAG b74d853ebe76 (cluster.py:446, __init__) 2026-10-30 18:51:40 [ 408 ] DEBUG : ENV SPARK_HOME /spark-3.3.2-bin-hadoop3 (cluster.py:446, __init__) 2026-10-30 18:51:40 [ 408 ] DEBUG : ENV LC_CTYPE C.UTF-8 (cluster.py:446, __init__) 2026-10-30 18:51:40 [ 408 ] DEBUG : ENV INTEGRATION_TESTS_RUN_ID 0 (cluster.py:446, __init__) 2026-10-30 18:51:40 [ 408 ] DEBUG : ENV WORKER_FREE_PORTS 30000 30001 30002 30003 30004 30005 30006 30007 30008 30009 30010 30011 30012 30013 30014 30015 30016 30017 30018 30019 30020 30021 30022 30023 30024 30025 30026 30027 30028 30029 30030 30031 30032 30033 30034 30035 30036 30037 30038 30039 30040 30041 30042 30043 30044 30045 30046 30047 30048 30049 (cluster.py:446, __init__) 2026-10-30 18:51:40 [ 408 ] DEBUG : ENV PYTEST_XDIST_TESTRUNUID fb1636694e5340dd8efa5d8334309001 (cluster.py:446, __init__) 2026-10-30 18:51:40 [ 408 ] DEBUG : ENV PYTEST_XDIST_WORKER gw0 (cluster.py:446, __init__) 2026-10-30 18:51:40 [ 408 ] DEBUG : ENV PYTEST_XDIST_WORKER_COUNT 10 (cluster.py:446, __init__) 2026-10-30 18:51:40 [ 408 ] DEBUG : ENV PYTEST_CURRENT_TEST test_storage_s3_queue/test.py::test_direct_select_file[unordered] (setup) (cluster.py:446, __init__) 2026-10-30 18:51:40 [ 408 ] DEBUG : CLUSTER INIT base_config_dir:/clickhouse-config (cluster.py:766, __init__) 2026-10-30 18:51:40 [ 408 ] DEBUG : clickhouse_start_command: clickhouse server --config-file=/etc/clickhouse-server/{main_config_file} --log-file=/var/log/clickhouse-server/clickhouse-server.log --errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log (cluster.py:1787, add_instance) 2026-10-30 18:51:40 [ 408 ] DEBUG : Setup Keeper (cluster.py:1059, setup_keeper_cmd) 2026-10-30 18:51:40 [ 408 ] DEBUG : Cluster name: project_name:rootteststorages3queue_gw0. Added instance name:instance tag:b74d853ebe76 base_cmd:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/.env', '--project-name', 'rootteststorages3queue_gw0', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/instance/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_azurite.yml'] docker_compose_yml_dir:/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/ (cluster.py:2081, add_instance) 2026-10-30 18:51:40 [ 408 ] DEBUG : clickhouse_start_command: clickhouse server --config-file=/etc/clickhouse-server/{main_config_file} --log-file=/var/log/clickhouse-server/clickhouse-server.log --errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log (cluster.py:1787, add_instance) 2026-10-30 18:51:40 [ 408 ] DEBUG : Cluster name: project_name:rootteststorages3queue_gw0. Added instance name:instance2 tag:b74d853ebe76 base_cmd:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/.env', '--project-name', 'rootteststorages3queue_gw0', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/instance/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_azurite.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/instance2/docker-compose.yml'] docker_compose_yml_dir:/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/ (cluster.py:2081, add_instance) 2026-10-30 18:51:40 [ 408 ] DEBUG : clickhouse_start_command: clickhouse server --config-file=/etc/clickhouse-server/{main_config_file} --log-file=/var/log/clickhouse-server/clickhouse-server.log --errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log (cluster.py:1787, add_instance) 2026-10-30 18:51:40 [ 408 ] DEBUG : Cluster name: project_name:rootteststorages3queue_gw0. Added instance name:old_instance tag:23.12 base_cmd:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/.env', '--project-name', 'rootteststorages3queue_gw0', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/instance/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_azurite.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/instance2/docker-compose.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/old_instance/docker-compose.yml'] docker_compose_yml_dir:/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/ (cluster.py:2081, add_instance) 2026-10-30 18:51:40 [ 408 ] DEBUG : clickhouse_start_command: clickhouse server --config-file=/etc/clickhouse-server/{main_config_file} --log-file=/var/log/clickhouse-server/clickhouse-server.log --errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log (cluster.py:1787, add_instance) 2026-10-30 18:51:40 [ 408 ] DEBUG : Cluster name: project_name:rootteststorages3queue_gw0. Added instance name:instance_too_many_parts tag:b74d853ebe76 base_cmd:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/.env', '--project-name', 'rootteststorages3queue_gw0', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/instance/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_azurite.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/instance2/docker-compose.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/old_instance/docker-compose.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/instance_too_many_parts/docker-compose.yml'] docker_compose_yml_dir:/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/ (cluster.py:2081, add_instance) 2026-10-30 18:51:40 [ 408 ] DEBUG : clickhouse_start_command: clickhouse server --config-file=/etc/clickhouse-server/{main_config_file} --log-file=/var/log/clickhouse-server/clickhouse-server.log --errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log (cluster.py:1787, add_instance) 2026-10-30 18:51:40 [ 408 ] DEBUG : Cluster name: project_name:rootteststorages3queue_gw0. Added instance name:instance_24.5 tag:24.5 base_cmd:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/.env', '--project-name', 'rootteststorages3queue_gw0', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/instance/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_azurite.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/instance2/docker-compose.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/old_instance/docker-compose.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/instance_too_many_parts/docker-compose.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/instance_24.5/docker-compose.yml'] docker_compose_yml_dir:/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/ (cluster.py:2081, add_instance) 2026-10-30 18:51:40 [ 408 ] INFO : Starting cluster... (test.py:143, started_cluster) 2026-10-30 18:51:40 [ 408 ] INFO : Running tests in /ClickHouse/tests/integration/test_storage_s3_queue/test.py (cluster.py:2790, start) 2026-10-30 18:51:40 [ 408 ] DEBUG : Cluster start called. is_up=False (cluster.py:2797, start) 2026-10-30 18:51:40 [ 408 ] DEBUG : Docker networks for project rootteststorages3queue_gw0 are NETWORK ID NAME DRIVER SCOPE (cluster.py:855, print_all_docker_pieces) 2026-10-30 18:51:40 [ 408 ] DEBUG : Docker containers for project rootteststorages3queue_gw0 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES (cluster.py:863, print_all_docker_pieces) 2026-10-30 18:51:40 [ 408 ] DEBUG : Docker volumes for project rootteststorages3queue_gw0 are DRIVER VOLUME NAME (cluster.py:871, print_all_docker_pieces) 2026-10-30 18:51:40 [ 408 ] DEBUG : Cleanup called (cluster.py:876, cleanup) 2026-10-30 18:51:40 [ 408 ] DEBUG : Docker networks for project rootteststorages3queue_gw0 are NETWORK ID NAME DRIVER SCOPE (cluster.py:855, print_all_docker_pieces) 2026-10-30 18:51:40 [ 408 ] DEBUG : Docker containers for project rootteststorages3queue_gw0 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES (cluster.py:863, print_all_docker_pieces) 2026-10-30 18:51:40 [ 408 ] DEBUG : Docker volumes for project rootteststorages3queue_gw0 are DRIVER VOLUME NAME (cluster.py:871, print_all_docker_pieces) 2026-10-30 18:51:40 [ 408 ] DEBUG : Command:docker container list --all --filter name='^/rootteststorages3queue_gw0_.*_1$' --format '{{.ID}}:{{.Names}}' (cluster.py:113, run_and_check) 2026-10-30 18:51:40 [ 408 ] DEBUG : Unstopped containers: {} (cluster.py:890, cleanup) 2026-10-30 18:51:40 [ 408 ] DEBUG : No running containers for project: rootteststorages3queue_gw0 (cluster.py:904, cleanup) 2026-10-30 18:51:40 [ 408 ] DEBUG : Trying to prune unused networks... (cluster.py:910, cleanup) 2026-10-30 18:51:40 [ 408 ] DEBUG : Trying to prune unused volumes... (cluster.py:937, cleanup) 2026-10-30 18:51:40 [ 408 ] DEBUG : Command:['docker volume ls | wc -l'] (cluster.py:113, run_and_check) 2026-10-30 18:51:40 [ 408 ] DEBUG : Stdout:1 (cluster.py:121, run_and_check) 2026-10-30 18:51:40 [ 408 ] DEBUG : Setup directory for instance: instance (cluster.py:2810, start) 2026-10-30 18:51:40 [ 408 ] DEBUG : Create directory for configuration generated in this helper (cluster.py:4538, create_dir) 2026-10-30 18:51:40 [ 408 ] DEBUG : Create directory for common tests configuration (cluster.py:4543, create_dir) 2026-10-30 18:51:40 [ 408 ] DEBUG : Copy common configuration from helpers (cluster.py:4563, create_dir) 2026-10-30 18:51:40 [ 408 ] DEBUG : Generate and write macros file (cluster.py:4606, create_dir) 2026-10-30 18:51:40 [ 408 ] DEBUG : Copy custom test config files ['/ClickHouse/tests/integration/test_storage_s3_queue/configs/zookeeper.xml', '/ClickHouse/tests/integration/test_storage_s3_queue/configs/s3queue_log.xml'] to /ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/instance/configs/config.d (cluster.py:4636, create_dir) 2026-10-30 18:51:40 [ 408 ] DEBUG : Setup database dir /ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/instance/database (cluster.py:4653, create_dir) 2026-10-30 18:51:40 [ 408 ] DEBUG : Setup logs dir /ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/instance/logs (cluster.py:4664, create_dir) 2026-10-30 18:51:40 [ 408 ] DEBUG : Entrypoint cmd: bash -c "trap 'pkill tail' INT TERM; clickhouse server --config-file=/etc/clickhouse-server/config.xml --log-file=/var/log/clickhouse-server/clickhouse-server.log --errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log --daemon; coproc tail -f /dev/null; wait $$!" (cluster.py:4750, create_dir) 2026-10-30 18:51:40 [ 408 ] DEBUG : Setup directory for instance: instance2 (cluster.py:2810, start) 2026-10-30 18:51:40 [ 408 ] DEBUG : Create directory for configuration generated in this helper (cluster.py:4538, create_dir) 2026-10-30 18:51:40 [ 408 ] DEBUG : Create directory for common tests configuration (cluster.py:4543, create_dir) 2026-10-30 18:51:40 [ 408 ] DEBUG : Copy common configuration from helpers (cluster.py:4563, create_dir) 2026-10-30 18:51:40 [ 408 ] DEBUG : Generate and write macros file (cluster.py:4606, create_dir) 2026-10-30 18:51:40 [ 408 ] DEBUG : Copy custom test config files ['/ClickHouse/tests/integration/test_storage_s3_queue/configs/s3queue_log.xml'] to /ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/instance2/configs/config.d (cluster.py:4636, create_dir) 2026-10-30 18:51:40 [ 408 ] DEBUG : Setup database dir /ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/instance2/database (cluster.py:4653, create_dir) 2026-10-30 18:51:40 [ 408 ] DEBUG : Setup logs dir /ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/instance2/logs (cluster.py:4664, create_dir) 2026-10-30 18:51:40 [ 408 ] DEBUG : Entrypoint cmd: bash -c "trap 'pkill tail' INT TERM; clickhouse server --config-file=/etc/clickhouse-server/config.xml --log-file=/var/log/clickhouse-server/clickhouse-server.log --errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log --daemon; coproc tail -f /dev/null; wait $$!" (cluster.py:4750, create_dir) 2026-10-30 18:51:40 [ 408 ] DEBUG : Setup directory for instance: old_instance (cluster.py:2810, start) 2026-10-30 18:51:40 [ 408 ] DEBUG : Create directory for configuration generated in this helper (cluster.py:4538, create_dir) 2026-10-30 18:51:40 [ 408 ] DEBUG : Create directory for common tests configuration (cluster.py:4543, create_dir) 2026-10-30 18:51:40 [ 408 ] DEBUG : Copy common configuration from helpers (cluster.py:4563, create_dir) 2026-10-30 18:51:40 [ 408 ] DEBUG : Generate and write macros file (cluster.py:4606, create_dir) 2026-10-30 18:51:40 [ 408 ] DEBUG : Copy custom test config files [] to /ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/old_instance/configs/config.d (cluster.py:4636, create_dir) 2026-10-30 18:51:40 [ 408 ] DEBUG : Setup database dir /ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/old_instance/database (cluster.py:4653, create_dir) 2026-10-30 18:51:40 [ 408 ] DEBUG : Setup logs dir /ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/old_instance/logs (cluster.py:4664, create_dir) 2026-10-30 18:51:40 [ 408 ] DEBUG : Entrypoint cmd: bash -c "trap 'pkill tail' INT TERM; clickhouse server --config-file=/etc/clickhouse-server/config.xml --log-file=/var/log/clickhouse-server/clickhouse-server.log --errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log --daemon; coproc tail -f /dev/null; wait $$!" (cluster.py:4750, create_dir) 2026-10-30 18:51:40 [ 408 ] DEBUG : Setup directory for instance: instance_too_many_parts (cluster.py:2810, start) 2026-10-30 18:51:40 [ 408 ] DEBUG : Create directory for configuration generated in this helper (cluster.py:4538, create_dir) 2026-10-30 18:51:40 [ 408 ] DEBUG : Create directory for common tests configuration (cluster.py:4543, create_dir) 2026-10-30 18:51:40 [ 408 ] DEBUG : Copy common configuration from helpers (cluster.py:4563, create_dir) 2026-10-30 18:51:40 [ 408 ] DEBUG : Generate and write macros file (cluster.py:4606, create_dir) 2026-10-30 18:51:40 [ 408 ] DEBUG : Copy custom test config files ['/ClickHouse/tests/integration/test_storage_s3_queue/configs/s3queue_log.xml', '/ClickHouse/tests/integration/test_storage_s3_queue/configs/merge_tree.xml'] to /ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/instance_too_many_parts/configs/config.d (cluster.py:4636, create_dir) 2026-10-30 18:51:40 [ 408 ] DEBUG : Setup database dir /ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/instance_too_many_parts/database (cluster.py:4653, create_dir) 2026-10-30 18:51:40 [ 408 ] DEBUG : Setup logs dir /ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/instance_too_many_parts/logs (cluster.py:4664, create_dir) 2026-10-30 18:51:40 [ 408 ] DEBUG : Entrypoint cmd: bash -c "trap 'pkill tail' INT TERM; clickhouse server --config-file=/etc/clickhouse-server/config.xml --log-file=/var/log/clickhouse-server/clickhouse-server.log --errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log --daemon; coproc tail -f /dev/null; wait $$!" (cluster.py:4750, create_dir) 2026-10-30 18:51:40 [ 408 ] DEBUG : Setup directory for instance: instance_24.5 (cluster.py:2810, start) 2026-10-30 18:51:40 [ 408 ] DEBUG : Create directory for configuration generated in this helper (cluster.py:4538, create_dir) 2026-10-30 18:51:40 [ 408 ] DEBUG : Create directory for common tests configuration (cluster.py:4543, create_dir) 2026-10-30 18:51:40 [ 408 ] DEBUG : Copy common configuration from helpers (cluster.py:4563, create_dir) 2026-10-30 18:51:40 [ 408 ] DEBUG : Generate and write macros file (cluster.py:4606, create_dir) 2026-10-30 18:51:40 [ 408 ] DEBUG : Copy custom test config files [] to /ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/instance_24.5/configs/config.d (cluster.py:4636, create_dir) 2026-10-30 18:51:40 [ 408 ] DEBUG : Setup database dir /ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/instance_24.5/database (cluster.py:4653, create_dir) 2026-10-30 18:51:40 [ 408 ] DEBUG : Setup logs dir /ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/instance_24.5/logs (cluster.py:4664, create_dir) 2026-10-30 18:51:40 [ 408 ] DEBUG : Entrypoint cmd: bash -c "trap 'pkill tail' INT TERM; clickhouse server --config-file=/etc/clickhouse-server/config.xml --log-file=/var/log/clickhouse-server/clickhouse-server.log --errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log --daemon; coproc tail -f /dev/null; wait $$!" (cluster.py:4750, create_dir) 2026-10-30 18:51:40 [ 408 ] DEBUG : Env {'ASAN_OPTIONS': 'use_sigaltstack=0', 'TSAN_OPTIONS': 'use_sigaltstack=0', 'CLICKHOUSE_WATCHDOG_ENABLE': '0', 'CLICKHOUSE_NATS_TLS_SECURE': '0', 'LLVM_PROFILE_FILE': '/var/lib/clickhouse/server_%h_%p_%m.profraw', 'keeper_binary': '/clickhouse', 'keeper_cmd_prefix': 'clickhouse keeper', 'image': 'altinityinfra/integration-test:b74d853ebe76', 'user': '0', 'keeper_fs': 'bind', 'keeper_logs_dir1': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/keeper1/log', 'keeper_config_dir1': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/keeper1/config', 'keeper_db_dir1': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/keeper1/coordination', 'keeper_logs_dir2': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/keeper2/log', 'keeper_config_dir2': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/keeper2/config', 'keeper_db_dir2': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/keeper2/coordination', 'keeper_logs_dir3': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/keeper3/log', 'keeper_config_dir3': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/keeper3/config', 'keeper_db_dir3': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/keeper3/coordination', 'MINIO_CERTS_DIR': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/minio/certs', 'MINIO_DATA_DIR': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/minio/data', 'MINIO_PORT': '9001', 'SSL_CERT_FILE': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/minio/certs/public.crt', 'AZURITE_PORT': '30000', 'AZURITE_STORAGE_ACCOUNT_URL': 'http://azurite1:30000/devstoreaccount1', 'AZURITE_CONNECTION_STRING': 'DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;BlobEndpoint=http://azurite1:30000/devstoreaccount1;'} stored in /ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/.env (cluster.py:86, _create_env_file) 2026-10-30 18:51:40 [ 408 ] DEBUG : Trying paths: ['/root/.docker/config.json', '/root/.dockercfg'] (config.py:21, find_config_file) 2026-10-30 18:51:40 [ 408 ] DEBUG : No config file found (config.py:28, find_config_file) 2026-10-30 18:51:40 [ 408 ] DEBUG : Trying paths: ['/root/.docker/config.json', '/root/.dockercfg'] (config.py:21, find_config_file) 2026-10-30 18:51:40 [ 408 ] DEBUG : No config file found (config.py:28, find_config_file) 2026-10-30 18:51:40 [ 408 ] DEBUG : http://localhost:None "GET /version HTTP/1.1" 200 825 (connectionpool.py:547, _make_request) 2026-10-30 18:51:40 [ 408 ] DEBUG : Command:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/.env', '--project-name', 'rootteststorages3queue_gw0', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/instance/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_azurite.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/instance2/docker-compose.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/old_instance/docker-compose.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/instance_too_many_parts/docker-compose.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/instance_24.5/docker-compose.yml', 'pull'] (cluster.py:113, run_and_check) 2026-10-30 18:51:45 [ 408 ] DEBUG : Stderr:Pulling azurite1 ... (cluster.py:123, run_and_check) 2026-10-30 18:51:45 [ 408 ] DEBUG : Stderr:Pulling zoo2 ... (cluster.py:123, run_and_check) 2026-10-30 18:51:45 [ 408 ] DEBUG : Stderr:Pulling proxy1 ... (cluster.py:123, run_and_check) 2026-10-30 18:51:45 [ 408 ] DEBUG : Stderr:Pulling zoo1 ... (cluster.py:123, run_and_check) 2026-10-30 18:51:45 [ 408 ] DEBUG : Stderr:Pulling proxy2 ... (cluster.py:123, run_and_check) 2026-10-30 18:51:45 [ 408 ] DEBUG : Stderr:Pulling resolver ... (cluster.py:123, run_and_check) 2026-10-30 18:51:45 [ 408 ] DEBUG : Stderr:Pulling minio1 ... (cluster.py:123, run_and_check) 2026-10-30 18:51:45 [ 408 ] DEBUG : Stderr:Pulling zoo3 ... (cluster.py:123, run_and_check) 2026-10-30 18:51:45 [ 408 ] DEBUG : Stderr:Pulling instance2 ... (cluster.py:123, run_and_check) 2026-10-30 18:51:45 [ 408 ] DEBUG : Stderr:Pulling instance_too_many_parts ... (cluster.py:123, run_and_check) 2026-10-30 18:51:45 [ 408 ] DEBUG : Stderr:Pulling old_instance ... (cluster.py:123, run_and_check) 2026-10-30 18:51:45 [ 408 ] DEBUG : Stderr:Pulling instance ... (cluster.py:123, run_and_check) 2026-10-30 18:51:45 [ 408 ] DEBUG : Stderr:Pulling instance_24.5 ... (cluster.py:123, run_and_check) 2026-10-30 18:51:45 [ 408 ] DEBUG : Stderr:Pulling instance_too_many_parts ... pulling from altinityinfra/integr... (cluster.py:123, run_and_check) 2026-10-30 18:51:45 [ 408 ] DEBUG : Stderr:Pulling proxy1 ... pulling from altinityinfra/s3-proxy (cluster.py:123, run_and_check) 2026-10-30 18:51:45 [ 408 ] DEBUG : Stderr:Pulling instance_24.5 ... pulling from clickhouse/clickhous... (cluster.py:123, run_and_check) 2026-10-30 18:51:45 [ 408 ] DEBUG : Stderr:Pulling minio1 ... pulling from clickhouse/minio-minio (cluster.py:123, run_and_check) 2026-10-30 18:51:45 [ 408 ] DEBUG : Stderr:Pulling instance ... pulling from altinityinfra/integr... (cluster.py:123, run_and_check) 2026-10-30 18:51:45 [ 408 ] DEBUG : Stderr:Pulling instance_too_many_parts ... digest: sha256:2ea9da862c215fd684... (cluster.py:123, run_and_check) 2026-10-30 18:51:45 [ 408 ] DEBUG : Stderr:Pulling instance_too_many_parts ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-10-30 18:51:45 [ 408 ] DEBUG : Stderr:Pulling instance_24.5 ... digest: sha256:f1155aa4adec32964c... (cluster.py:123, run_and_check) 2026-10-30 18:51:45 [ 408 ] DEBUG : Stderr:Pulling instance_24.5 ... status: image is up to date for c... (cluster.py:123, run_and_check) 2026-10-30 18:51:45 [ 408 ] DEBUG : Stderr:Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... (cluster.py:123, run_and_check) 2026-10-30 18:51:45 [ 408 ] DEBUG : Stderr:Pulling proxy1 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-10-30 18:51:45 [ 408 ] DEBUG : Stderr:Pulling minio1 ... digest: sha256:9eea83a4e1425067e7... (cluster.py:123, run_and_check) 2026-10-30 18:51:45 [ 408 ] DEBUG : Stderr:Pulling minio1 ... status: image is up to date for c... (cluster.py:123, run_and_check) 2026-10-30 18:51:45 [ 408 ] DEBUG : Stderr:Pulling instance ... digest: sha256:2ea9da862c215fd684... (cluster.py:123, run_and_check) 2026-10-30 18:51:45 [ 408 ] DEBUG : Stderr:Pulling instance ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-10-30 18:51:45 [ 408 ] DEBUG : Stderr:Pulling minio1 ... done (cluster.py:123, run_and_check) 2026-10-30 18:51:45 [ 408 ] DEBUG : Stderr:Pulling instance_24.5 ... done (cluster.py:123, run_and_check) 2026-10-30 18:51:45 [ 408 ] DEBUG : Stderr:Pulling instance_too_many_parts ... done (cluster.py:123, run_and_check) 2026-10-30 18:51:45 [ 408 ] DEBUG : Stderr:Pulling proxy1 ... done (cluster.py:123, run_and_check) 2026-10-30 18:51:45 [ 408 ] DEBUG : Stderr:Pulling instance ... done (cluster.py:123, run_and_check) 2026-10-30 18:51:45 [ 408 ] DEBUG : Stderr:Pulling instance2 ... pulling from altinityinfra/integr... (cluster.py:123, run_and_check) 2026-10-30 18:51:45 [ 408 ] DEBUG : Stderr:Pulling zoo1 ... pulling from altinityinfra/integr... (cluster.py:123, run_and_check) 2026-10-30 18:51:45 [ 408 ] DEBUG : Stderr:Pulling zoo3 ... pulling from altinityinfra/integr... (cluster.py:123, run_and_check) 2026-10-30 18:51:45 [ 408 ] DEBUG : Stderr:Pulling old_instance ... pulling from clickhouse/clickhous... (cluster.py:123, run_and_check) 2026-10-30 18:51:45 [ 408 ] DEBUG : Stderr:Pulling azurite1 ... pulling from altinityinfra/azurite (cluster.py:123, run_and_check) 2026-10-30 18:51:45 [ 408 ] DEBUG : Stderr:Pulling instance2 ... digest: sha256:2ea9da862c215fd684... (cluster.py:123, run_and_check) 2026-10-30 18:51:45 [ 408 ] DEBUG : Stderr:Pulling instance2 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-10-30 18:51:45 [ 408 ] DEBUG : Stderr:Pulling azurite1 ... digest: sha256:dae2a5f96553962901... (cluster.py:123, run_and_check) 2026-10-30 18:51:45 [ 408 ] DEBUG : Stderr:Pulling azurite1 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-10-30 18:51:45 [ 408 ] DEBUG : Stderr:Pulling instance2 ... done (cluster.py:123, run_and_check) 2026-10-30 18:51:45 [ 408 ] DEBUG : Stderr:Pulling zoo1 ... digest: sha256:2ea9da862c215fd684... (cluster.py:123, run_and_check) 2026-10-30 18:51:45 [ 408 ] DEBUG : Stderr:Pulling azurite1 ... done (cluster.py:123, run_and_check) 2026-10-30 18:51:45 [ 408 ] DEBUG : Stderr:Pulling zoo1 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-10-30 18:51:45 [ 408 ] DEBUG : Stderr:Pulling zoo1 ... done (cluster.py:123, run_and_check) 2026-10-30 18:51:45 [ 408 ] DEBUG : Stderr:Pulling old_instance ... digest: sha256:5d0802b64c09f6d2c6... (cluster.py:123, run_and_check) 2026-10-30 18:51:45 [ 408 ] DEBUG : Stderr:Pulling old_instance ... status: image is up to date for c... (cluster.py:123, run_and_check) 2026-10-30 18:51:45 [ 408 ] DEBUG : Stderr:Pulling zoo3 ... digest: sha256:2ea9da862c215fd684... (cluster.py:123, run_and_check) 2026-10-30 18:51:45 [ 408 ] DEBUG : Stderr:Pulling old_instance ... done (cluster.py:123, run_and_check) 2026-10-30 18:51:45 [ 408 ] DEBUG : Stderr:Pulling zoo3 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-10-30 18:51:45 [ 408 ] DEBUG : Stderr:Pulling zoo3 ... done (cluster.py:123, run_and_check) 2026-10-30 18:51:45 [ 408 ] DEBUG : Stderr:Pulling proxy2 ... pulling from altinityinfra/s3-proxy (cluster.py:123, run_and_check) 2026-10-30 18:51:45 [ 408 ] DEBUG : Stderr:Pulling zoo2 ... pulling from altinityinfra/integr... (cluster.py:123, run_and_check) 2026-10-30 18:51:45 [ 408 ] DEBUG : Stderr:Pulling resolver ... pulling from altinityinfra/python... (cluster.py:123, run_and_check) 2026-10-30 18:51:45 [ 408 ] DEBUG : Stderr:Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... (cluster.py:123, run_and_check) 2026-10-30 18:51:45 [ 408 ] DEBUG : Stderr:Pulling proxy2 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-10-30 18:51:45 [ 408 ] DEBUG : Stderr:Pulling proxy2 ... done (cluster.py:123, run_and_check) 2026-10-30 18:51:45 [ 408 ] DEBUG : Stderr:Pulling zoo2 ... digest: sha256:2ea9da862c215fd684... (cluster.py:123, run_and_check) 2026-10-30 18:51:45 [ 408 ] DEBUG : Stderr:Pulling zoo2 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-10-30 18:51:45 [ 408 ] DEBUG : Stderr:Pulling zoo2 ... done (cluster.py:123, run_and_check) 2026-10-30 18:51:45 [ 408 ] DEBUG : Stderr:Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... (cluster.py:123, run_and_check) 2026-10-30 18:51:45 [ 408 ] DEBUG : Stderr:Pulling resolver ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-10-30 18:51:45 [ 408 ] DEBUG : Stderr:Pulling resolver ... done (cluster.py:123, run_and_check) 2026-10-30 18:51:45 [ 408 ] DEBUG : Setup ZooKeeper (cluster.py:2853, start) 2026-10-30 18:51:45 [ 408 ] DEBUG : Creating internal ZooKeeper dirs: ['/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/keeper1/log', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/keeper1/config', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/keeper1/coordination', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/keeper2/log', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/keeper2/config', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/keeper2/coordination', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/keeper3/log', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/keeper3/config', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/keeper3/coordination'] (cluster.py:2854, start) 2026-10-30 18:51:45 [ 408 ] DEBUG : Command:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/.env', '--project-name', 'rootteststorages3queue_gw0', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml', '--verbose', 'up', '-d'] (cluster.py:113, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.config.config.find: Using configuration files: /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.cli.docker_client.get_client: docker-compose version 1.29.2, build unknown (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:docker-py version: (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:CPython version: 3.10.12 (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:OpenSSL version: OpenSSL 3.0.2 15 Mar 2022 (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.cli.docker_client.get_client: Docker base_url: http+docker://localhost (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.cli.docker_client.get_client: Docker version: Platform={'Name': 'Docker Engine - Community'}, Components=[{'Name': 'Engine', 'Version': '23.0.6', 'Details': {'ApiVersion': '1.42', 'Arch': 'arm64', 'BuildTime': '2023-05-05T21:18:33.000000000+00:00', 'Experimental': 'false', 'GitCommit': '9dbdbd4', 'GoVersion': 'go1.19.9', 'KernelVersion': '5.15.0-198-generic', 'MinAPIVersion': '1.12', 'Os': 'linux'}}, {'Name': 'containerd', 'Version': 'v2.2.6', 'Details': {'GitCommit': '11ce9d5f3c68c941867e82890e93e815c1304f1b'}}, {'Name': 'runc', 'Version': '1.3.6', 'Details': {'GitCommit': 'v1.3.6-0-g491b69ba'}}, {'Name': 'docker-init', 'Version': '0.19.0', 'Details': {'GitCommit': 'de40ad0'}}], Version=23.0.6, ApiVersion=1.42, MinAPIVersion=1.12, GitCommit=9dbdbd4, GoVersion=go1.19.9, Os=linux, Arch=arm64, KernelVersion=5.15.0-198-generic, BuildTime=2023-05-05T21:18:33.000000000+00:00 (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_network <- ('rootteststorages3queuegw0_default') (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker info <- () (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker info -> {'Architecture': 'aarch64', (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'BridgeNfIp6tables': True, (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'BridgeNfIptables': True, (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'CPUSet': True, (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'CPUShares': True, (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'CgroupDriver': 'cgroupfs', (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'CgroupVersion': '2', (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'ContainerdCommit': {'Expected': '11ce9d5f3c68c941867e82890e93e815c1304f1b', (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'ID': '11ce9d5f3c68c941867e82890e93e815c1304f1b'}, (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'Containers': 16, (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_network <- ('rootteststorages3queue_gw0_default') (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.network.ensure: Creating network "rootteststorages3queue_gw0_default" with the default driver (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_network <- (name='rootteststorages3queue_gw0_default', driver=None, options=None, ipam=None, internal=False, enable_ipv6=False, labels={'com.docker.compose.project': 'rootteststorages3queue_gw0', 'com.docker.compose.network': 'default', 'com.docker.compose.version': '1.29.2'}, attachable=True, check_duplicate=True) (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_network -> {'Id': '7e2f6eee4392e35e2b6d933e9f84746bfac62d40d371e87dea56d810738e88d0', (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'Warning': ''} (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=False, filters={'label': ['com.docker.compose.project=rootteststorages3queue_gw0', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=False, filters={'label': ['com.docker.compose.project=rootteststorages3queuegw0', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue_gw0', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queuegw0', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue_gw0', 'com.docker.compose.service=zoo1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queuegw0', 'com.docker.compose.service=zoo1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue_gw0', 'com.docker.compose.service=zoo2', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queuegw0', 'com.docker.compose.service=zoo2', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue_gw0', 'com.docker.compose.service=zoo3', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queuegw0', 'com.docker.compose.service=zoo3', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/integration-test:b74d853ebe76') (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'arm64', (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'Author': '', (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'AttachStderr': False, (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'Cmd': ['/bin/sh', '-c', 'sleep 1'], (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'Entrypoint': None, (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/integration-test:b74d853ebe76') (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'arm64', (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'Author': '', (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'AttachStderr': False, (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'Cmd': ['/bin/sh', '-c', 'sleep 1'], (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'Entrypoint': None, (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/integration-test:b74d853ebe76') (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'arm64', (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'Author': '', (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'AttachStderr': False, (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'Cmd': ['/bin/sh', '-c', 'sleep 1'], (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'Entrypoint': None, (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue_gw0', 'com.docker.compose.service=zoo1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queuegw0', 'com.docker.compose.service=zoo1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue_gw0', 'com.docker.compose.service=zoo2', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queuegw0', 'com.docker.compose.service=zoo2', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue_gw0', 'com.docker.compose.service=zoo3', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queuegw0', 'com.docker.compose.service=zoo3', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: {, , } (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Starting producer thread for (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue_gw0', 'com.docker.compose.service=zoo1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Starting producer thread for (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue_gw0', 'com.docker.compose.service=zoo3', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Starting producer thread for (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queuegw0', 'com.docker.compose.service=zoo1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue_gw0', 'com.docker.compose.service=zoo2', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:Creating rootteststorages3queue_gw0_zoo1_1 ... (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queuegw0', 'com.docker.compose.service=zoo3', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: {ServiceName(project='rootteststorages3queue_gw0', service='zoo1', number=1)} (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Starting producer thread for ServiceName(project='rootteststorages3queue_gw0', service='zoo1', number=1) (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queuegw0', 'com.docker.compose.service=zoo2', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:Creating rootteststorages3queue_gw0_zoo3_1 ... (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: {ServiceName(project='rootteststorages3queue_gw0', service='zoo3', number=1)} (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Starting producer thread for ServiceName(project='rootteststorages3queue_gw0', service='zoo3', number=1) (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:Creating rootteststorages3queue_gw0_zoo2_1 ... (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: {ServiceName(project='rootteststorages3queue_gw0', service='zoo2', number=1)} (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Starting producer thread for ServiceName(project='rootteststorages3queue_gw0', service='zoo2', number=1) (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/integration-test:b74d853ebe76') (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/integration-test:b74d853ebe76') (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/integration-test:b74d853ebe76') (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'arm64', (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'Author': '', (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'AttachStderr': False, (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'Cmd': ['/bin/sh', '-c', 'sleep 1'], (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'Entrypoint': None, (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/integration-test:b74d853ebe76') (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'arm64', (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'Author': '', (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'AttachStderr': False, (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'Cmd': ['/bin/sh', '-c', 'sleep 1'], (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'Entrypoint': None, (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'arm64', (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'Author': '', (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'AttachStderr': False, (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'Cmd': ['/bin/sh', '-c', 'sleep 1'], (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'Entrypoint': None, (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/integration-test:b74d853ebe76') (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/integration-test:b74d853ebe76') (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'arm64', (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'Author': '', (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'AttachStderr': False, (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'Cmd': ['/bin/sh', '-c', 'sleep 1'], (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'Entrypoint': None, (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.service.build_container_labels: Added config hash: d67b04e660080bb63060f5ab40a2e0db813487233f0c9bf7d963197427fbd59c (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_host_config <- (links=[], port_bindings={}, binds=[], volumes_from=[], privileged=False, network_mode='rootteststorages3queue_gw0_default', devices=None, device_requests=None, dns=None, dns_opt=['attempts:2', 'timeout:1', 'inet6', 'rotate'], dns_search=None, restart_policy={'Name': 'always', 'MaximumRetryCount': 0}, runtime=None, cap_add=['SYS_PTRACE', 'NET_ADMIN', 'IPC_LOCK', 'SYS_NICE'], cap_drop=None, mem_limit=None, mem_reservation=None, memswap_limit=None, ulimits=None, log_config={'Type': '', 'Config': {}}, extra_hosts=None, read_only=None, pid_mode=None, security_opt=['label:disable'], ipc_mode=None, cgroup_parent=None, cpu_quota=None, shm_size=None, sysctls=None, pids_limit=None, tmpfs=None, oom_kill_disable=None, oom_score_adj=None, mem_swappiness=None, group_add=None, userns_mode=None, init=None, init_path=None, isolation=None, cpu_count=None, cpu_percent=None, nano_cpus=None, volume_driver=None, cpuset_cpus=None, cpu_shares=None, storage_opt=None, blkio_weight=None, blkio_weight_device=None, device_read_bps=None, device_read_iops=None, device_write_bps=None, device_write_iops=None, mounts=[{'Target': '/usr/bin/clickhouse-keeper', 'Source': '/clickhouse', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/lib/clickhouse', 'Source': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/keeper1/coordination', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/log/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/keeper1/log', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/etc/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/keeper1/config', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/lib/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/keeper1/coordination', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/usr/bin/clickhouse', 'Source': '/clickhouse', 'Type': 'bind', 'ReadOnly': None}], device_cgroup_rules=None, cpu_period=None, cpu_rt_period=None, cpu_rt_runtime=None) (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_host_config -> {'Binds': [], (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'CapAdd': ['SYS_PTRACE', 'NET_ADMIN', 'IPC_LOCK', 'SYS_NICE'], (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'DnsOptions': ['attempts:2', 'timeout:1', 'inet6', 'rotate'], (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'Links': [], (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'LogConfig': {'Config': {}, 'Type': ''}, (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'Mounts': [{'ReadOnly': None, (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'Source': '/clickhouse', (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'Target': '/usr/bin/clickhouse-keeper', (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'Type': 'bind'}, (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: {'ReadOnly': None, (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_container <- (entrypoint='clickhouse keeper --config=/etc/clickhouse-keeper/keeper_config1.xml --log-file=/var/log/clickhouse-keeper/clickhouse-keeper.log --errorlog-file=/var/log/clickhouse-keeper/clickhouse-keeper.err.log', image='altinityinfra/integration-test:b74d853ebe76', user='0', volumes={}, name='rootteststorages3queue_gw0_zoo1_1', detach=True, environment=[], labels={'com.docker.compose.project': 'rootteststorages3queue_gw0', 'com.docker.compose.service': 'zoo1', 'com.docker.compose.oneoff': 'False', 'com.docker.compose.project.working_dir': '/ClickHouse/tests/integration/compose', 'com.docker.compose.project.config_files': '/ClickHouse/tests/integration/compose/docker_compose_keeper.yml', 'com.docker.compose.project.environment_file': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/.env', 'com.docker.compose.container-number': '1', 'com.docker.compose.version': '1.29.2', 'com.docker.compose.config-hash': 'd67b04e660080bb63060f5ab40a2e0db813487233f0c9bf7d963197427fbd59c'}, host_config={'NetworkMode': 'rootteststorages3queue_gw0_default', 'RestartPolicy': {'Name': 'always', 'MaximumRetryCount': 0}, 'CapAdd': ['SYS_PTRACE', 'NET_ADMIN', 'IPC_LOCK', 'SYS_NICE'], 'DnsOptions': ['attempts:2', 'timeout:1', 'inet6', 'rotate'], 'SecurityOpt': ['label:disable'], 'VolumesFrom': [], 'Binds': [], 'PortBindings': {}, 'Links': [], 'LogConfig': {'Type': '', 'Config': {}}, 'Mounts': [{'Target': '/usr/bin/clickhouse-keeper', 'Source': '/clickhouse', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/lib/clickhouse', 'Source': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/keeper1/coordination', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/log/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/keeper1/log', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/etc/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/keeper1/config', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/lib/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/keeper1/coordination', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/usr/bin/clickhouse', 'Source': '/clickhouse', 'Type': 'bind', 'ReadOnly': None}]}, networking_config={'EndpointsConfig': {'rootteststorages3queue_gw0_default': {'Aliases': ['zoo1'], 'IPAMConfig': {}}}}) (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'arm64', (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'Author': '', (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'AttachStderr': False, (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'Cmd': ['/bin/sh', '-c', 'sleep 1'], (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'Entrypoint': None, (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.service.build_container_labels: Added config hash: b50202438ed60bf94e731b1816ce66cecefbd9cd363fba8d206b261271994880 (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_host_config <- (links=[], port_bindings={}, binds=[], volumes_from=[], privileged=False, network_mode='rootteststorages3queue_gw0_default', devices=None, device_requests=None, dns=None, dns_opt=['attempts:2', 'timeout:1', 'inet6', 'rotate'], dns_search=None, restart_policy={'Name': 'always', 'MaximumRetryCount': 0}, runtime=None, cap_add=['SYS_PTRACE', 'NET_ADMIN', 'IPC_LOCK', 'SYS_NICE'], cap_drop=None, mem_limit=None, mem_reservation=None, memswap_limit=None, ulimits=None, log_config={'Type': '', 'Config': {}}, extra_hosts=None, read_only=None, pid_mode=None, security_opt=['label:disable'], ipc_mode=None, cgroup_parent=None, cpu_quota=None, shm_size=None, sysctls=None, pids_limit=None, tmpfs=None, oom_kill_disable=None, oom_score_adj=None, mem_swappiness=None, group_add=None, userns_mode=None, init=None, init_path=None, isolation=None, cpu_count=None, cpu_percent=None, nano_cpus=None, volume_driver=None, cpuset_cpus=None, cpu_shares=None, storage_opt=None, blkio_weight=None, blkio_weight_device=None, device_read_bps=None, device_read_iops=None, device_write_bps=None, device_write_iops=None, mounts=[{'Target': '/usr/bin/clickhouse', 'Source': '/clickhouse', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/log/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/keeper2/log', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/usr/bin/clickhouse-keeper', 'Source': '/clickhouse', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/lib/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/keeper2/coordination', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/lib/clickhouse', 'Source': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/keeper1/coordination', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/etc/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/keeper2/config', 'Type': 'bind', 'ReadOnly': None}], device_cgroup_rules=None, cpu_period=None, cpu_rt_period=None, cpu_rt_runtime=None) (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_host_config -> {'Binds': [], (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'CapAdd': ['SYS_PTRACE', 'NET_ADMIN', 'IPC_LOCK', 'SYS_NICE'], (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'DnsOptions': ['attempts:2', 'timeout:1', 'inet6', 'rotate'], (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'Links': [], (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'LogConfig': {'Config': {}, 'Type': ''}, (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'Mounts': [{'ReadOnly': None, (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'Source': '/clickhouse', (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'Target': '/usr/bin/clickhouse', (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'Type': 'bind'}, (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: {'ReadOnly': None, (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_container <- (entrypoint='clickhouse keeper --config=/etc/clickhouse-keeper/keeper_config2.xml --log-file=/var/log/clickhouse-keeper/clickhouse-keeper.log --errorlog-file=/var/log/clickhouse-keeper/clickhouse-keeper.err.log', image='altinityinfra/integration-test:b74d853ebe76', user='0', volumes={}, name='rootteststorages3queue_gw0_zoo2_1', detach=True, environment=[], labels={'com.docker.compose.project': 'rootteststorages3queue_gw0', 'com.docker.compose.service': 'zoo2', 'com.docker.compose.oneoff': 'False', 'com.docker.compose.project.working_dir': '/ClickHouse/tests/integration/compose', 'com.docker.compose.project.config_files': '/ClickHouse/tests/integration/compose/docker_compose_keeper.yml', 'com.docker.compose.project.environment_file': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/.env', 'com.docker.compose.container-number': '1', 'com.docker.compose.version': '1.29.2', 'com.docker.compose.config-hash': 'b50202438ed60bf94e731b1816ce66cecefbd9cd363fba8d206b261271994880'}, host_config={'NetworkMode': 'rootteststorages3queue_gw0_default', 'RestartPolicy': {'Name': 'always', 'MaximumRetryCount': 0}, 'CapAdd': ['SYS_PTRACE', 'NET_ADMIN', 'IPC_LOCK', 'SYS_NICE'], 'DnsOptions': ['attempts:2', 'timeout:1', 'inet6', 'rotate'], 'SecurityOpt': ['label:disable'], 'VolumesFrom': [], 'Binds': [], 'PortBindings': {}, 'Links': [], 'LogConfig': {'Type': '', 'Config': {}}, 'Mounts': [{'Target': '/usr/bin/clickhouse', 'Source': '/clickhouse', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/log/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/keeper2/log', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/usr/bin/clickhouse-keeper', 'Source': '/clickhouse', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/lib/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/keeper2/coordination', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/lib/clickhouse', 'Source': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/keeper1/coordination', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/etc/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/keeper2/config', 'Type': 'bind', 'ReadOnly': None}]}, networking_config={'EndpointsConfig': {'rootteststorages3queue_gw0_default': {'Aliases': ['zoo2'], 'IPAMConfig': {}}}}) (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'arm64', (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'Author': '', (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'AttachStderr': False, (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'Cmd': ['/bin/sh', '-c', 'sleep 1'], (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'Entrypoint': None, (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.service.build_container_labels: Added config hash: deb9a5aed43d8fd58a36aefd3043d247f1039432a859ce6d588183ff55208e74 (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_host_config <- (links=[], port_bindings={}, binds=[], volumes_from=[], privileged=False, network_mode='rootteststorages3queue_gw0_default', devices=None, device_requests=None, dns=None, dns_opt=['attempts:2', 'timeout:1', 'inet6', 'rotate'], dns_search=None, restart_policy={'Name': 'always', 'MaximumRetryCount': 0}, runtime=None, cap_add=['SYS_PTRACE', 'NET_ADMIN', 'IPC_LOCK', 'SYS_NICE'], cap_drop=None, mem_limit=None, mem_reservation=None, memswap_limit=None, ulimits=None, log_config={'Type': '', 'Config': {}}, extra_hosts=None, read_only=None, pid_mode=None, security_opt=['label:disable'], ipc_mode=None, cgroup_parent=None, cpu_quota=None, shm_size=None, sysctls=None, pids_limit=None, tmpfs=None, oom_kill_disable=None, oom_score_adj=None, mem_swappiness=None, group_add=None, userns_mode=None, init=None, init_path=None, isolation=None, cpu_count=None, cpu_percent=None, nano_cpus=None, volume_driver=None, cpuset_cpus=None, cpu_shares=None, storage_opt=None, blkio_weight=None, blkio_weight_device=None, device_read_bps=None, device_read_iops=None, device_write_bps=None, device_write_iops=None, mounts=[{'Target': '/usr/bin/clickhouse-keeper', 'Source': '/clickhouse', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/etc/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/keeper3/config', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/log/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/keeper3/log', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/lib/clickhouse', 'Source': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/keeper1/coordination', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/lib/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/keeper3/coordination', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/usr/bin/clickhouse', 'Source': '/clickhouse', 'Type': 'bind', 'ReadOnly': None}], device_cgroup_rules=None, cpu_period=None, cpu_rt_period=None, cpu_rt_runtime=None) (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_host_config -> {'Binds': [], (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'CapAdd': ['SYS_PTRACE', 'NET_ADMIN', 'IPC_LOCK', 'SYS_NICE'], (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'DnsOptions': ['attempts:2', 'timeout:1', 'inet6', 'rotate'], (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'Links': [], (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'LogConfig': {'Config': {}, 'Type': ''}, (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'Mounts': [{'ReadOnly': None, (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'Source': '/clickhouse', (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'Target': '/usr/bin/clickhouse-keeper', (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'Type': 'bind'}, (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: {'ReadOnly': None, (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_container <- (entrypoint='clickhouse keeper --config=/etc/clickhouse-keeper/keeper_config3.xml --log-file=/var/log/clickhouse-keeper/clickhouse-keeper.log --errorlog-file=/var/log/clickhouse-keeper/clickhouse-keeper.err.log', image='altinityinfra/integration-test:b74d853ebe76', user='0', volumes={}, name='rootteststorages3queue_gw0_zoo3_1', detach=True, environment=[], labels={'com.docker.compose.project': 'rootteststorages3queue_gw0', 'com.docker.compose.service': 'zoo3', 'com.docker.compose.oneoff': 'False', 'com.docker.compose.project.working_dir': '/ClickHouse/tests/integration/compose', 'com.docker.compose.project.config_files': '/ClickHouse/tests/integration/compose/docker_compose_keeper.yml', 'com.docker.compose.project.environment_file': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/.env', 'com.docker.compose.container-number': '1', 'com.docker.compose.version': '1.29.2', 'com.docker.compose.config-hash': 'deb9a5aed43d8fd58a36aefd3043d247f1039432a859ce6d588183ff55208e74'}, host_config={'NetworkMode': 'rootteststorages3queue_gw0_default', 'RestartPolicy': {'Name': 'always', 'MaximumRetryCount': 0}, 'CapAdd': ['SYS_PTRACE', 'NET_ADMIN', 'IPC_LOCK', 'SYS_NICE'], 'DnsOptions': ['attempts:2', 'timeout:1', 'inet6', 'rotate'], 'SecurityOpt': ['label:disable'], 'VolumesFrom': [], 'Binds': [], 'PortBindings': {}, 'Links': [], 'LogConfig': {'Type': '', 'Config': {}}, 'Mounts': [{'Target': '/usr/bin/clickhouse-keeper', 'Source': '/clickhouse', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/etc/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/keeper3/config', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/log/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/keeper3/log', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/lib/clickhouse', 'Source': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/keeper1/coordination', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/var/lib/clickhouse-keeper', 'Source': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/keeper3/coordination', 'Type': 'bind', 'ReadOnly': None}, {'Target': '/usr/bin/clickhouse', 'Source': '/clickhouse', 'Type': 'bind', 'ReadOnly': None}]}, networking_config={'EndpointsConfig': {'rootteststorages3queue_gw0_default': {'Aliases': ['zoo3'], 'IPAMConfig': {}}}}) (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_container -> {'Id': '71024ce75a4999e2cbf31df30572e641bb088f736f7fe01259a50b8796c1c29c', (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'Warnings': []} (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container <- ('71024ce75a4999e2cbf31df30572e641bb088f736f7fe01259a50b8796c1c29c') (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_container -> {'Id': '3abd8efabb519533a4cc04dbe8e1d41a098b78e994a45e4f9022dfd90d231806', (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'Warnings': []} (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container <- ('3abd8efabb519533a4cc04dbe8e1d41a098b78e994a45e4f9022dfd90d231806') (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_container -> {'Id': '1f40c6eafa1401644cbd49e44d1c7cd8b5c7f792429edd711392c265c4b2def4', (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'Warnings': []} (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container <- ('1f40c6eafa1401644cbd49e44d1c7cd8b5c7f792429edd711392c265c4b2def4') (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container -> {'AppArmorProfile': '', (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'Args': ['keeper', (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: '--config=/etc/clickhouse-keeper/keeper_config1.xml', (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: '--log-file=/var/log/clickhouse-keeper/clickhouse-keeper.log', (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: '--errorlog-file=/var/log/clickhouse-keeper/clickhouse-keeper.err.log'], (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'Config': {'AttachStderr': False, (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'Cmd': None, (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker disconnect_container_from_network <- ('3abd8efabb519533a4cc04dbe8e1d41a098b78e994a45e4f9022dfd90d231806', 'rootteststorages3queue_gw0_default') (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container -> {'AppArmorProfile': '', (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'Args': ['keeper', (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: '--config=/etc/clickhouse-keeper/keeper_config2.xml', (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: '--log-file=/var/log/clickhouse-keeper/clickhouse-keeper.log', (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: '--errorlog-file=/var/log/clickhouse-keeper/clickhouse-keeper.err.log'], (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'Config': {'AttachStderr': False, (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'Cmd': None, (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker disconnect_container_from_network <- ('71024ce75a4999e2cbf31df30572e641bb088f736f7fe01259a50b8796c1c29c', 'rootteststorages3queue_gw0_default') (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container -> {'AppArmorProfile': '', (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'Args': ['keeper', (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: '--config=/etc/clickhouse-keeper/keeper_config3.xml', (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: '--log-file=/var/log/clickhouse-keeper/clickhouse-keeper.log', (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: '--errorlog-file=/var/log/clickhouse-keeper/clickhouse-keeper.err.log'], (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'Config': {'AttachStderr': False, (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'Cmd': None, (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker disconnect_container_from_network <- ('1f40c6eafa1401644cbd49e44d1c7cd8b5c7f792429edd711392c265c4b2def4', 'rootteststorages3queue_gw0_default') (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker disconnect_container_from_network -> None (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker connect_container_to_network <- ('3abd8efabb519533a4cc04dbe8e1d41a098b78e994a45e4f9022dfd90d231806', 'rootteststorages3queue_gw0_default', aliases=['3abd8efabb51', 'zoo1'], ipv4_address=None, ipv6_address=None, links=[], link_local_ips=None) (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker disconnect_container_from_network -> None (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker connect_container_to_network <- ('71024ce75a4999e2cbf31df30572e641bb088f736f7fe01259a50b8796c1c29c', 'rootteststorages3queue_gw0_default', aliases=['71024ce75a49', 'zoo2'], ipv4_address=None, ipv6_address=None, links=[], link_local_ips=None) (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker disconnect_container_from_network -> None (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker connect_container_to_network <- ('1f40c6eafa1401644cbd49e44d1c7cd8b5c7f792429edd711392c265c4b2def4', 'rootteststorages3queue_gw0_default', aliases=['zoo3', '1f40c6eafa14'], ipv4_address=None, ipv6_address=None, links=[], link_local_ips=None) (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker connect_container_to_network -> None (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker start <- ('3abd8efabb519533a4cc04dbe8e1d41a098b78e994a45e4f9022dfd90d231806') (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker connect_container_to_network -> None (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker start <- ('1f40c6eafa1401644cbd49e44d1c7cd8b5c7f792429edd711392c265c4b2def4') (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker connect_container_to_network -> None (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker start <- ('71024ce75a4999e2cbf31df30572e641bb088f736f7fe01259a50b8796c1c29c') (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker start -> None (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.parallel.parallel_execute_iter: Finished processing: ServiceName(project='rootteststorages3queue_gw0', service='zoo3', number=1) (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:Creating rootteststorages3queue_gw0_zoo3_1 ... done (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.parallel.parallel_execute_iter: Finished processing: (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker start -> None (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.parallel.parallel_execute_iter: Finished processing: ServiceName(project='rootteststorages3queue_gw0', service='zoo2', number=1) (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:Creating rootteststorages3queue_gw0_zoo2_1 ... done (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.parallel.parallel_execute_iter: Finished processing: (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker start -> None (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.parallel.parallel_execute_iter: Finished processing: ServiceName(project='rootteststorages3queue_gw0', service='zoo1', number=1) (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:Creating rootteststorages3queue_gw0_zoo1_1 ... done (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.parallel.parallel_execute_iter: Finished processing: (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:51:50 [ 408 ] DEBUG : Wait ZooKeeper to start (cluster.py:2506, wait_zookeeper_to_start) 2026-10-30 18:51:50 [ 408 ] DEBUG : get_instance_ip instance_name=zoo1 (cluster.py:2137, get_instance_ip) 2026-10-30 18:51:50 [ 408 ] DEBUG : http://localhost:None "GET /v1.42/containers/rootteststorages3queue_gw0_zoo1_1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-10-30 18:51:50 [ 408 ] DEBUG : get_kazoo_client: zoo1, ip:172.16.10.2, port:2181, use_ssl:False (cluster.py:3290, get_kazoo_client) 2026-10-30 18:51:50 [ 408 ] INFO : Connecting to 172.16.10.2(172.16.10.2):2181, use_ssl: False (connection.py:650, _connect) 2026-10-30 18:51:50 [ 408 ] WARNING : Connection dropped: socket connection error: Connection refused (connection.py:622, _connect_attempt) 2026-10-30 18:51:50 [ 408 ] INFO : Connecting to 172.16.10.2(172.16.10.2):2181, use_ssl: False (connection.py:650, _connect) 2026-10-30 18:51:50 [ 408 ] WARNING : Connection dropped: socket connection error: Connection refused (connection.py:622, _connect_attempt) 2026-10-30 18:51:50 [ 408 ] INFO : Connecting to 172.16.10.2(172.16.10.2):2181, use_ssl: False (connection.py:650, _connect) 2026-10-30 18:51:50 [ 408 ] WARNING : Connection dropped: socket connection error: Connection refused (connection.py:622, _connect_attempt) 2026-10-30 18:51:50 [ 408 ] INFO : Connecting to 172.16.10.2(172.16.10.2):2181, use_ssl: False (connection.py:650, _connect) 2026-10-30 18:51:50 [ 408 ] WARNING : Connection dropped: socket connection error: Connection refused (connection.py:622, _connect_attempt) 2026-10-30 18:51:52 [ 408 ] INFO : Connecting to 172.16.10.2(172.16.10.2):2181, use_ssl: False (connection.py:650, _connect) 2026-10-30 18:51:52 [ 408 ] WARNING : Connection dropped: socket connection error: Connection refused (connection.py:622, _connect_attempt) 2026-10-30 18:51:55 [ 408 ] INFO : Connecting to 172.16.10.2(172.16.10.2):2181, use_ssl: False (connection.py:650, _connect) 2026-10-30 18:51:55 [ 408 ] DEBUG : Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=10000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) (connection.py:312, _submit) 2026-10-30 18:51:55 [ 408 ] INFO : Zookeeper connection established, state: CONNECTED (client.py:532, _session_callback) 2026-10-30 18:51:55 [ 408 ] DEBUG : Sending request(xid=1): GetChildren(path='/', watcher=None) (connection.py:312, _submit) 2026-10-30 18:51:55 [ 408 ] DEBUG : Received response(xid=1): ['keeper'] (connection.py:410, _read_response) 2026-10-30 18:51:55 [ 408 ] DEBUG : Sending request(xid=2): Close() (connection.py:312, _submit) 2026-10-30 18:51:55 [ 408 ] WARNING : Connection dropped: socket connection broken (connection.py:622, _connect_attempt) 2026-10-30 18:51:55 [ 408 ] WARNING : Transition to CONNECTING (connection.py:626, _connect_attempt) 2026-10-30 18:51:55 [ 408 ] INFO : Zookeeper connection lost (client.py:543, _session_callback) 2026-10-30 18:51:55 [ 408 ] WARNING : Failed connecting to Zookeeper within the connection retry policy. (connection.py:515, zk_loop) 2026-10-30 18:51:55 [ 408 ] INFO : Zookeeper session closed, state: CLOSED (client.py:537, _session_callback) 2026-10-30 18:51:55 [ 408 ] DEBUG : get_instance_ip instance_name=zoo2 (cluster.py:2137, get_instance_ip) 2026-10-30 18:51:55 [ 408 ] DEBUG : http://localhost:None "GET /v1.42/containers/rootteststorages3queue_gw0_zoo2_1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-10-30 18:51:55 [ 408 ] DEBUG : get_kazoo_client: zoo2, ip:172.16.10.4, port:2181, use_ssl:False (cluster.py:3290, get_kazoo_client) 2026-10-30 18:51:55 [ 408 ] INFO : Connecting to 172.16.10.4(172.16.10.4):2181, use_ssl: False (connection.py:650, _connect) 2026-10-30 18:51:55 [ 408 ] DEBUG : Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=10000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) (connection.py:312, _submit) 2026-10-30 18:51:55 [ 408 ] INFO : Zookeeper connection established, state: CONNECTED (client.py:532, _session_callback) 2026-10-30 18:51:55 [ 408 ] DEBUG : Sending request(xid=1): GetChildren(path='/', watcher=None) (connection.py:312, _submit) 2026-10-30 18:51:55 [ 408 ] DEBUG : Received response(xid=1): ['keeper'] (connection.py:410, _read_response) 2026-10-30 18:51:55 [ 408 ] DEBUG : Sending request(xid=2): Close() (connection.py:312, _submit) 2026-10-30 18:51:55 [ 408 ] WARNING : Connection dropped: socket connection broken (connection.py:622, _connect_attempt) 2026-10-30 18:51:55 [ 408 ] WARNING : Transition to CONNECTING (connection.py:626, _connect_attempt) 2026-10-30 18:51:55 [ 408 ] INFO : Zookeeper connection lost (client.py:543, _session_callback) 2026-10-30 18:51:55 [ 408 ] WARNING : Failed connecting to Zookeeper within the connection retry policy. (connection.py:515, zk_loop) 2026-10-30 18:51:55 [ 408 ] INFO : Zookeeper session closed, state: CLOSED (client.py:537, _session_callback) 2026-10-30 18:51:55 [ 408 ] DEBUG : get_instance_ip instance_name=zoo3 (cluster.py:2137, get_instance_ip) 2026-10-30 18:51:55 [ 408 ] DEBUG : http://localhost:None "GET /v1.42/containers/rootteststorages3queue_gw0_zoo3_1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-10-30 18:51:55 [ 408 ] DEBUG : get_kazoo_client: zoo3, ip:172.16.10.3, port:2181, use_ssl:False (cluster.py:3290, get_kazoo_client) 2026-10-30 18:51:55 [ 408 ] INFO : Connecting to 172.16.10.3(172.16.10.3):2181, use_ssl: False (connection.py:650, _connect) 2026-10-30 18:51:55 [ 408 ] DEBUG : Sending request(xid=None): Connect(protocol_version=0, last_zxid_seen=0, time_out=10000, session_id=0, passwd=b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', read_only=None) (connection.py:312, _submit) 2026-10-30 18:51:55 [ 408 ] INFO : Zookeeper connection established, state: CONNECTED (client.py:532, _session_callback) 2026-10-30 18:51:55 [ 408 ] DEBUG : Sending request(xid=1): GetChildren(path='/', watcher=None) (connection.py:312, _submit) 2026-10-30 18:51:55 [ 408 ] DEBUG : Received response(xid=1): ['keeper'] (connection.py:410, _read_response) 2026-10-30 18:51:55 [ 408 ] DEBUG : Sending request(xid=2): Close() (connection.py:312, _submit) 2026-10-30 18:51:55 [ 408 ] WARNING : Connection dropped: socket connection broken (connection.py:622, _connect_attempt) 2026-10-30 18:51:55 [ 408 ] WARNING : Transition to CONNECTING (connection.py:626, _connect_attempt) 2026-10-30 18:51:55 [ 408 ] INFO : Zookeeper connection lost (client.py:543, _session_callback) 2026-10-30 18:51:55 [ 408 ] WARNING : Failed connecting to Zookeeper within the connection retry policy. (connection.py:515, zk_loop) 2026-10-30 18:51:55 [ 408 ] INFO : Zookeeper session closed, state: CLOSED (client.py:537, _session_callback) 2026-10-30 18:51:55 [ 408 ] DEBUG : All instances of ZooKeeper Secure started (cluster.py:2521, wait_zookeeper_nodes_to_start) 2026-10-30 18:51:55 [ 408 ] INFO : Trying to create Minio instance by command docker-compose --env-file /ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/.env --project-name rootteststorages3queue_gw0 --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml --verbose up -d (cluster.py:3091, start) 2026-10-30 18:51:55 [ 408 ] DEBUG : Command:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/.env', '--project-name', 'rootteststorages3queue_gw0', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--verbose', 'up', '-d'] (cluster.py:113, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.config.config.find: Using configuration files: /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.docker_client.get_client: docker-compose version 1.29.2, build unknown (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:docker-py version: (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:CPython version: 3.10.12 (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:OpenSSL version: OpenSSL 3.0.2 15 Mar 2022 (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.docker_client.get_client: Docker base_url: http+docker://localhost (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.docker_client.get_client: Docker version: Platform={'Name': 'Docker Engine - Community'}, Components=[{'Name': 'Engine', 'Version': '23.0.6', 'Details': {'ApiVersion': '1.42', 'Arch': 'arm64', 'BuildTime': '2023-05-05T21:18:33.000000000+00:00', 'Experimental': 'false', 'GitCommit': '9dbdbd4', 'GoVersion': 'go1.19.9', 'KernelVersion': '5.15.0-198-generic', 'MinAPIVersion': '1.12', 'Os': 'linux'}}, {'Name': 'containerd', 'Version': 'v2.2.6', 'Details': {'GitCommit': '11ce9d5f3c68c941867e82890e93e815c1304f1b'}}, {'Name': 'runc', 'Version': '1.3.6', 'Details': {'GitCommit': 'v1.3.6-0-g491b69ba'}}, {'Name': 'docker-init', 'Version': '0.19.0', 'Details': {'GitCommit': 'de40ad0'}}], Version=23.0.6, ApiVersion=1.42, MinAPIVersion=1.12, GitCommit=9dbdbd4, GoVersion=go1.19.9, Os=linux, Arch=arm64, KernelVersion=5.15.0-198-generic, BuildTime=2023-05-05T21:18:33.000000000+00:00 (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_network <- ('rootteststorages3queuegw0_default') (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_volume <- ('rootteststorages3queuegw0_data1-1') (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker info <- () (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker info -> {'Architecture': 'aarch64', (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'BridgeNfIp6tables': True, (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'BridgeNfIptables': True, (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'CPUSet': True, (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'CPUShares': True, (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'CgroupDriver': 'cgroupfs', (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'CgroupVersion': '2', (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'ContainerdCommit': {'Expected': '11ce9d5f3c68c941867e82890e93e815c1304f1b', (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'ID': '11ce9d5f3c68c941867e82890e93e815c1304f1b'}, (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Containers': 37, (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_network <- ('rootteststorages3queue_gw0_default') (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_network -> {'Attachable': True, (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'ConfigFrom': {'Network': ''}, (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'ConfigOnly': False, (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Containers': {'1f40c6eafa1401644cbd49e44d1c7cd8b5c7f792429edd711392c265c4b2def4': {'EndpointID': 'eb17be299b0aaa6738af7d25794dde23266cb63850519c573a8bfced50065494', (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'IPv4Address': '172.16.10.3/24', (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'IPv6Address': '', (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'MacAddress': '02:42:ac:10:0a:03', (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Name': 'rootteststorages3queue_gw0_zoo3_1'}, (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: '3abd8efabb519533a4cc04dbe8e1d41a098b78e994a45e4f9022dfd90d231806': {'EndpointID': '74129e2d62ac300658e125fdcb26ff54a349b46e4d241d4b6f9ae597a2743955', (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'IPv4Address': '172.16.10.2/24', (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_volume <- ('rootteststorages3queue_gw0_data1-1') (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.volume.initialize: Creating volume "rootteststorages3queue_gw0_data1-1" with default driver (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_volume <- ('rootteststorages3queue_gw0_data1-1', None, None, labels={'com.docker.compose.project': 'rootteststorages3queue_gw0', 'com.docker.compose.volume': 'data1-1', 'com.docker.compose.version': '1.29.2'}) (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_volume -> {'CreatedAt': '2026-10-30T18:51:56Z', (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Driver': 'local', (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Labels': {'com.docker.compose.project': 'rootteststorages3queue_gw0', (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'com.docker.compose.version': '1.29.2', (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'com.docker.compose.volume': 'data1-1'}, (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Mountpoint': '/var/lib/docker/volumes/rootteststorages3queue_gw0_data1-1/_data', (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Name': 'rootteststorages3queue_gw0_data1-1', (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Options': None, (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Scope': 'local'} (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=False, filters={'label': ['com.docker.compose.project=rootteststorages3queue_gw0', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 3 items) (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue_gw0', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 3 items) (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container <- ('71024ce75a4999e2cbf31df30572e641bb088f736f7fe01259a50b8796c1c29c') (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container -> {'AppArmorProfile': '', (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Args': ['keeper', (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: '--config=/etc/clickhouse-keeper/keeper_config2.xml', (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: '--log-file=/var/log/clickhouse-keeper/clickhouse-keeper.log', (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: '--errorlog-file=/var/log/clickhouse-keeper/clickhouse-keeper.err.log'], (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Config': {'AttachStderr': False, (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Cmd': None, (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container <- ('3abd8efabb519533a4cc04dbe8e1d41a098b78e994a45e4f9022dfd90d231806') (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container -> {'AppArmorProfile': '', (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Args': ['keeper', (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: '--config=/etc/clickhouse-keeper/keeper_config1.xml', (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: '--log-file=/var/log/clickhouse-keeper/clickhouse-keeper.log', (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: '--errorlog-file=/var/log/clickhouse-keeper/clickhouse-keeper.err.log'], (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Config': {'AttachStderr': False, (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Cmd': None, (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container <- ('1f40c6eafa1401644cbd49e44d1c7cd8b5c7f792429edd711392c265c4b2def4') (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container -> {'AppArmorProfile': '', (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Args': ['keeper', (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: '--config=/etc/clickhouse-keeper/keeper_config3.xml', (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: '--log-file=/var/log/clickhouse-keeper/clickhouse-keeper.log', (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: '--errorlog-file=/var/log/clickhouse-keeper/clickhouse-keeper.err.log'], (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Config': {'AttachStderr': False, (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Cmd': None, (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.project.find_orphan_containers: Found orphan containers (rootteststorages3queue_gw0_zoo2_1, rootteststorages3queue_gw0_zoo1_1, rootteststorages3queue_gw0_zoo3_1) for this project. If you removed or renamed this service in your compose file, you can run this command with the --remove-orphans flag to clean it up. (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue_gw0', 'com.docker.compose.service=proxy1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queuegw0', 'com.docker.compose.service=proxy1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue_gw0', 'com.docker.compose.service=proxy2', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queuegw0', 'com.docker.compose.service=proxy2', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue_gw0', 'com.docker.compose.service=minio1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queuegw0', 'com.docker.compose.service=minio1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue_gw0', 'com.docker.compose.service=resolver', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queuegw0', 'com.docker.compose.service=resolver', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/s3-proxy') (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'arm64', (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Author': '', (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'AttachStderr': False, (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Cmd': ['/run.sh'], (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Entrypoint': ['/docker-entrypoint.sh'], (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/s3-proxy') (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'arm64', (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Author': '', (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'AttachStderr': False, (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Cmd': ['/run.sh'], (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Entrypoint': ['/docker-entrypoint.sh'], (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('clickhouse/minio-minio:RELEASE.2024-07-31T05-46-26Z') (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'arm64', (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Author': '', (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'AttachStderr': False, (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Cmd': ['minio'], (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Entrypoint': ['/usr/bin/docker-entrypoint.sh'], (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/python-bottle:caad4729259e') (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'arm64', (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Author': '', (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'AttachStderr': False, (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Cmd': ['python3'], (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Entrypoint': None, (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue_gw0', 'com.docker.compose.service=proxy1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queuegw0', 'com.docker.compose.service=proxy1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue_gw0', 'com.docker.compose.service=proxy2', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queuegw0', 'com.docker.compose.service=proxy2', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.project._get_convergence_plans: minio1 has upstream changes (proxy1, proxy2) (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'status': ['created', 'exited'], 'label': ['com.docker.compose.project=rootteststorages3queue_gw0', 'com.docker.compose.service=minio1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'status': ['created', 'exited'], 'label': ['com.docker.compose.project=rootteststorages3queuegw0', 'com.docker.compose.service=minio1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=False, filters={'label': ['com.docker.compose.project=rootteststorages3queue_gw0', 'com.docker.compose.service=minio1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=False, filters={'label': ['com.docker.compose.project=rootteststorages3queuegw0', 'com.docker.compose.service=minio1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue_gw0', 'com.docker.compose.service=minio1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queuegw0', 'com.docker.compose.service=minio1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.project._get_convergence_plans: resolver has upstream changes (proxy1, proxy2) (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'status': ['created', 'exited'], 'label': ['com.docker.compose.project=rootteststorages3queue_gw0', 'com.docker.compose.service=resolver', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'status': ['created', 'exited'], 'label': ['com.docker.compose.project=rootteststorages3queuegw0', 'com.docker.compose.service=resolver', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=False, filters={'label': ['com.docker.compose.project=rootteststorages3queue_gw0', 'com.docker.compose.service=resolver', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=False, filters={'label': ['com.docker.compose.project=rootteststorages3queuegw0', 'com.docker.compose.service=resolver', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue_gw0', 'com.docker.compose.service=resolver', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queuegw0', 'com.docker.compose.service=resolver', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: {, , , } (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Starting producer thread for (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue_gw0', 'com.docker.compose.service=proxy2', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Starting producer thread for (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue_gw0', 'com.docker.compose.service=proxy1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queuegw0', 'com.docker.compose.service=proxy2', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:Creating rootteststorages3queue_gw0_proxy2_1 ... (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queuegw0', 'com.docker.compose.service=proxy1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: {ServiceName(project='rootteststorages3queue_gw0', service='proxy2', number=1)} (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Starting producer thread for ServiceName(project='rootteststorages3queue_gw0', service='proxy2', number=1) (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/s3-proxy') (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:Creating rootteststorages3queue_gw0_proxy1_1 ... (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: {ServiceName(project='rootteststorages3queue_gw0', service='proxy1', number=1)} (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Starting producer thread for ServiceName(project='rootteststorages3queue_gw0', service='proxy1', number=1) (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/s3-proxy') (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'arm64', (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Author': '', (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'AttachStderr': False, (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Cmd': ['/run.sh'], (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Entrypoint': ['/docker-entrypoint.sh'], (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'arm64', (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Author': '', (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'AttachStderr': False, (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Cmd': ['/run.sh'], (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Entrypoint': ['/docker-entrypoint.sh'], (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/s3-proxy') (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/s3-proxy') (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'arm64', (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Author': '', (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'AttachStderr': False, (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Cmd': ['/run.sh'], (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Entrypoint': ['/docker-entrypoint.sh'], (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.service.build_container_labels: Added config hash: 83955d752b258c25aab8cbc51e773d66012280ec97adfa158748a204aaf2a65e (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_host_config <- (links=[], port_bindings={}, binds=[], volumes_from=[], privileged=False, network_mode='rootteststorages3queue_gw0_default', devices=None, device_requests=None, dns=None, dns_opt=None, dns_search=None, restart_policy=None, runtime=None, cap_add=None, cap_drop=None, mem_limit=None, mem_reservation=None, memswap_limit=None, ulimits=None, log_config={'Type': '', 'Config': {}}, extra_hosts=None, read_only=None, pid_mode=None, security_opt=None, ipc_mode=None, cgroup_parent=None, cpu_quota=None, shm_size=None, sysctls=None, pids_limit=None, tmpfs=None, oom_kill_disable=None, oom_score_adj=None, mem_swappiness=None, group_add=None, userns_mode=None, init=None, init_path=None, isolation=None, cpu_count=None, cpu_percent=None, nano_cpus=None, volume_driver=None, cpuset_cpus=None, cpu_shares=None, storage_opt=None, blkio_weight=None, blkio_weight_device=None, device_read_bps=None, device_read_iops=None, device_write_bps=None, device_write_iops=None, mounts=None, device_cgroup_rules=None, cpu_period=None, cpu_rt_period=None, cpu_rt_runtime=None) (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_host_config -> {'Binds': [], (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Links': [], (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'LogConfig': {'Config': {}, 'Type': ''}, (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'NetworkMode': 'rootteststorages3queue_gw0_default', (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'PortBindings': {}, (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'VolumesFrom': []} (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_container <- (image='altinityinfra/s3-proxy', volumes={}, name='rootteststorages3queue_gw0_proxy1_1', detach=True, ports=['8080', '80', '443'], environment=[], labels={'com.docker.compose.project': 'rootteststorages3queue_gw0', 'com.docker.compose.service': 'proxy1', 'com.docker.compose.oneoff': 'False', 'com.docker.compose.project.working_dir': '/ClickHouse/tests/integration/compose', 'com.docker.compose.project.config_files': '/ClickHouse/tests/integration/compose/docker_compose_minio.yml', 'com.docker.compose.project.environment_file': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/.env', 'com.docker.compose.container-number': '1', 'com.docker.compose.version': '1.29.2', 'com.docker.compose.config-hash': '83955d752b258c25aab8cbc51e773d66012280ec97adfa158748a204aaf2a65e'}, host_config={'NetworkMode': 'rootteststorages3queue_gw0_default', 'VolumesFrom': [], 'Binds': [], 'PortBindings': {}, 'Links': [], 'LogConfig': {'Type': '', 'Config': {}}}, networking_config={'EndpointsConfig': {'rootteststorages3queue_gw0_default': {'Aliases': ['proxy1'], 'IPAMConfig': {}}}}) (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'arm64', (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Author': '', (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'AttachStderr': False, (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Cmd': ['/run.sh'], (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Entrypoint': ['/docker-entrypoint.sh'], (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.service.build_container_labels: Added config hash: 83955d752b258c25aab8cbc51e773d66012280ec97adfa158748a204aaf2a65e (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_host_config <- (links=[], port_bindings={}, binds=[], volumes_from=[], privileged=False, network_mode='rootteststorages3queue_gw0_default', devices=None, device_requests=None, dns=None, dns_opt=None, dns_search=None, restart_policy=None, runtime=None, cap_add=None, cap_drop=None, mem_limit=None, mem_reservation=None, memswap_limit=None, ulimits=None, log_config={'Type': '', 'Config': {}}, extra_hosts=None, read_only=None, pid_mode=None, security_opt=None, ipc_mode=None, cgroup_parent=None, cpu_quota=None, shm_size=None, sysctls=None, pids_limit=None, tmpfs=None, oom_kill_disable=None, oom_score_adj=None, mem_swappiness=None, group_add=None, userns_mode=None, init=None, init_path=None, isolation=None, cpu_count=None, cpu_percent=None, nano_cpus=None, volume_driver=None, cpuset_cpus=None, cpu_shares=None, storage_opt=None, blkio_weight=None, blkio_weight_device=None, device_read_bps=None, device_read_iops=None, device_write_bps=None, device_write_iops=None, mounts=None, device_cgroup_rules=None, cpu_period=None, cpu_rt_period=None, cpu_rt_runtime=None) (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_host_config -> {'Binds': [], (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Links': [], (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'LogConfig': {'Config': {}, 'Type': ''}, (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'NetworkMode': 'rootteststorages3queue_gw0_default', (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'PortBindings': {}, (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'VolumesFrom': []} (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_container <- (image='altinityinfra/s3-proxy', volumes={}, name='rootteststorages3queue_gw0_proxy2_1', detach=True, ports=['8080', '80', '443'], environment=[], labels={'com.docker.compose.project': 'rootteststorages3queue_gw0', 'com.docker.compose.service': 'proxy2', 'com.docker.compose.oneoff': 'False', 'com.docker.compose.project.working_dir': '/ClickHouse/tests/integration/compose', 'com.docker.compose.project.config_files': '/ClickHouse/tests/integration/compose/docker_compose_minio.yml', 'com.docker.compose.project.environment_file': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/.env', 'com.docker.compose.container-number': '1', 'com.docker.compose.version': '1.29.2', 'com.docker.compose.config-hash': '83955d752b258c25aab8cbc51e773d66012280ec97adfa158748a204aaf2a65e'}, host_config={'NetworkMode': 'rootteststorages3queue_gw0_default', 'VolumesFrom': [], 'Binds': [], 'PortBindings': {}, 'Links': [], 'LogConfig': {'Type': '', 'Config': {}}}, networking_config={'EndpointsConfig': {'rootteststorages3queue_gw0_default': {'Aliases': ['proxy2'], 'IPAMConfig': {}}}}) (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: {, } (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_container -> {'Id': '70924df5ac9ac27415657dd1665d0e0d9caeb6741e354c44a13b490d29d373d9', (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Warnings': []} (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container <- ('70924df5ac9ac27415657dd1665d0e0d9caeb6741e354c44a13b490d29d373d9') (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_container -> {'Id': '55184b41c92593eda8d504918421ddd7632b28c3a8ad2ddda9dd6373681d1194', (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Warnings': []} (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container <- ('55184b41c92593eda8d504918421ddd7632b28c3a8ad2ddda9dd6373681d1194') (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container -> {'AppArmorProfile': '', (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Args': ['/run.sh'], (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Config': {'AttachStderr': False, (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Cmd': ['/run.sh'], (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Entrypoint': ['/docker-entrypoint.sh'], (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Env': ['PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin', (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'NGINX_VERSION=1.27.5', (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker disconnect_container_from_network <- ('55184b41c92593eda8d504918421ddd7632b28c3a8ad2ddda9dd6373681d1194', 'rootteststorages3queue_gw0_default') (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container -> {'AppArmorProfile': '', (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Args': ['/run.sh'], (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Config': {'AttachStderr': False, (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Cmd': ['/run.sh'], (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Entrypoint': ['/docker-entrypoint.sh'], (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Env': ['PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin', (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'NGINX_VERSION=1.27.5', (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker disconnect_container_from_network <- ('70924df5ac9ac27415657dd1665d0e0d9caeb6741e354c44a13b490d29d373d9', 'rootteststorages3queue_gw0_default') (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: {, } (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker disconnect_container_from_network -> None (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker connect_container_to_network <- ('55184b41c92593eda8d504918421ddd7632b28c3a8ad2ddda9dd6373681d1194', 'rootteststorages3queue_gw0_default', aliases=['proxy2', '55184b41c925'], ipv4_address=None, ipv6_address=None, links=[], link_local_ips=None) (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker disconnect_container_from_network -> None (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker connect_container_to_network <- ('70924df5ac9ac27415657dd1665d0e0d9caeb6741e354c44a13b490d29d373d9', 'rootteststorages3queue_gw0_default', aliases=['70924df5ac9a', 'proxy1'], ipv4_address=None, ipv6_address=None, links=[], link_local_ips=None) (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker connect_container_to_network -> None (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker start <- ('55184b41c92593eda8d504918421ddd7632b28c3a8ad2ddda9dd6373681d1194') (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker connect_container_to_network -> None (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker start <- ('70924df5ac9ac27415657dd1665d0e0d9caeb6741e354c44a13b490d29d373d9') (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: {, } (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: {, } (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: {, } (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: {, } (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: {, } (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: {, } (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: {, } (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker start -> None (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.parallel.parallel_execute_iter: Finished processing: ServiceName(project='rootteststorages3queue_gw0', service='proxy2', number=1) (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:Creating rootteststorages3queue_gw0_proxy2_1 ... done (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.parallel.parallel_execute_iter: Finished processing: (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: {, } (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: {, } (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker start -> None (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.parallel.parallel_execute_iter: Finished processing: ServiceName(project='rootteststorages3queue_gw0', service='proxy1', number=1) (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:Creating rootteststorages3queue_gw0_proxy1_1 ... done (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.parallel.parallel_execute_iter: Finished processing: (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: {, } (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Starting producer thread for (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue_gw0', 'com.docker.compose.service=minio1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Starting producer thread for (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queuegw0', 'com.docker.compose.service=minio1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue_gw0', 'com.docker.compose.service=resolver', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queuegw0', 'com.docker.compose.service=resolver', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:Creating rootteststorages3queue_gw0_resolver_1 ... (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: {ServiceName(project='rootteststorages3queue_gw0', service='resolver', number=1)} (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Starting producer thread for ServiceName(project='rootteststorages3queue_gw0', service='resolver', number=1) (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:Creating rootteststorages3queue_gw0_minio1_1 ... (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: {ServiceName(project='rootteststorages3queue_gw0', service='minio1', number=1)} (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Starting producer thread for ServiceName(project='rootteststorages3queue_gw0', service='minio1', number=1) (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/python-bottle:caad4729259e') (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('clickhouse/minio-minio:RELEASE.2024-07-31T05-46-26Z') (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'arm64', (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Author': '', (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'AttachStderr': False, (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Cmd': ['minio'], (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Entrypoint': ['/usr/bin/docker-entrypoint.sh'], (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'arm64', (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Author': '', (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'AttachStderr': False, (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Cmd': ['python3'], (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Entrypoint': None, (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/python-bottle:caad4729259e') (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('clickhouse/minio-minio:RELEASE.2024-07-31T05-46-26Z') (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'arm64', (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Author': '', (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'AttachStderr': False, (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Cmd': ['python3'], (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Entrypoint': None, (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'arm64', (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Author': '', (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'AttachStderr': False, (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Cmd': ['minio'], (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Entrypoint': ['/usr/bin/docker-entrypoint.sh'], (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.service.build_container_labels: Added config hash: 15fbc2e720a805dc427f06653b7c954d1877773fcd7fae997d8818818b77e713 (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.service.build_container_labels: Added config hash: 67b240acc3399fd5f3e80d9be0ea0188b596dde1c99b0cdc68bc70b597309126 (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_host_config <- (links=[], port_bindings={}, binds=[], volumes_from=[], privileged=False, network_mode='rootteststorages3queue_gw0_default', devices=None, device_requests=None, dns=None, dns_opt=None, dns_search=None, restart_policy=None, runtime=None, cap_add=None, cap_drop=None, mem_limit=None, mem_reservation=None, memswap_limit=None, ulimits=None, log_config={'Type': '', 'Config': {}}, extra_hosts=None, read_only=None, pid_mode=None, security_opt=None, ipc_mode=None, cgroup_parent=None, cpu_quota=None, shm_size=None, sysctls=None, pids_limit=None, tmpfs=None, oom_kill_disable=None, oom_score_adj=None, mem_swappiness=None, group_add=None, userns_mode=None, init=None, init_path=None, isolation=None, cpu_count=None, cpu_percent=None, nano_cpus=None, volume_driver=None, cpuset_cpus=None, cpu_shares=None, storage_opt=None, blkio_weight=None, blkio_weight_device=None, device_read_bps=None, device_read_iops=None, device_write_bps=None, device_write_iops=None, mounts=None, device_cgroup_rules=None, cpu_period=None, cpu_rt_period=None, cpu_rt_runtime=None) (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_host_config <- (links=[], port_bindings={}, binds=['rootteststorages3queue_gw0_data1-1:/data1:rw', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/minio/certs:/certs:rw'], volumes_from=[], privileged=False, network_mode='rootteststorages3queue_gw0_default', devices=None, device_requests=None, dns=None, dns_opt=None, dns_search=None, restart_policy=None, runtime=None, cap_add=None, cap_drop=None, mem_limit=None, mem_reservation=None, memswap_limit=None, ulimits=None, log_config={'Type': '', 'Config': {}}, extra_hosts=None, read_only=None, pid_mode=None, security_opt=None, ipc_mode=None, cgroup_parent=None, cpu_quota=None, shm_size=None, sysctls=None, pids_limit=None, tmpfs=None, oom_kill_disable=None, oom_score_adj=None, mem_swappiness=None, group_add=None, userns_mode=None, init=None, init_path=None, isolation=None, cpu_count=None, cpu_percent=None, nano_cpus=None, volume_driver=None, cpuset_cpus=None, cpu_shares=None, storage_opt=None, blkio_weight=None, blkio_weight_device=None, device_read_bps=None, device_read_iops=None, device_write_bps=None, device_write_iops=None, mounts=None, device_cgroup_rules=None, cpu_period=None, cpu_rt_period=None, cpu_rt_runtime=None) (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_host_config -> {'Binds': [], (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Links': [], (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'LogConfig': {'Config': {}, 'Type': ''}, (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'NetworkMode': 'rootteststorages3queue_gw0_default', (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'PortBindings': {}, (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'VolumesFrom': []} (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_host_config -> {'Binds': ['rootteststorages3queue_gw0_data1-1:/data1:rw', (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/minio/certs:/certs:rw'], (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Links': [], (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'LogConfig': {'Config': {}, 'Type': ''}, (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'NetworkMode': 'rootteststorages3queue_gw0_default', (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'PortBindings': {}, (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'VolumesFrom': []} (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_container <- (image='altinityinfra/python-bottle:caad4729259e', tty=True, volumes={}, name='rootteststorages3queue_gw0_resolver_1', detach=True, ports=['8080'], environment=[], labels={'com.docker.compose.project': 'rootteststorages3queue_gw0', 'com.docker.compose.service': 'resolver', 'com.docker.compose.oneoff': 'False', 'com.docker.compose.project.working_dir': '/ClickHouse/tests/integration/compose', 'com.docker.compose.project.config_files': '/ClickHouse/tests/integration/compose/docker_compose_minio.yml', 'com.docker.compose.project.environment_file': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/.env', 'com.docker.compose.container-number': '1', 'com.docker.compose.version': '1.29.2', 'com.docker.compose.config-hash': '15fbc2e720a805dc427f06653b7c954d1877773fcd7fae997d8818818b77e713'}, host_config={'NetworkMode': 'rootteststorages3queue_gw0_default', 'VolumesFrom': [], 'Binds': [], 'PortBindings': {}, 'Links': [], 'LogConfig': {'Type': '', 'Config': {}}}, networking_config={'EndpointsConfig': {'rootteststorages3queue_gw0_default': {'Aliases': ['resolver'], 'IPAMConfig': {}}}}) (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_container <- (command='server --console-address 127.0.0.1:19001 --address :9001 --certs-dir /certs /data1-1', environment=['MINIO_ACCESS_KEY=minio', 'MINIO_SECRET_KEY=minio123', 'MINIO_PROMETHEUS_AUTH_TYPE=public'], image='clickhouse/minio-minio:RELEASE.2024-07-31T05-46-26Z', volumes={'/data1': {}, '/certs': {}}, name='rootteststorages3queue_gw0_minio1_1', detach=True, ports=['9001'], labels={'com.docker.compose.project': 'rootteststorages3queue_gw0', 'com.docker.compose.service': 'minio1', 'com.docker.compose.oneoff': 'False', 'com.docker.compose.project.working_dir': '/ClickHouse/tests/integration/compose', 'com.docker.compose.project.config_files': '/ClickHouse/tests/integration/compose/docker_compose_minio.yml', 'com.docker.compose.project.environment_file': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/.env', 'com.docker.compose.container-number': '1', 'com.docker.compose.version': '1.29.2', 'com.docker.compose.config-hash': '67b240acc3399fd5f3e80d9be0ea0188b596dde1c99b0cdc68bc70b597309126'}, host_config={'NetworkMode': 'rootteststorages3queue_gw0_default', 'VolumesFrom': [], 'Binds': ['rootteststorages3queue_gw0_data1-1:/data1:rw', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/minio/certs:/certs:rw'], 'PortBindings': {}, 'Links': [], 'LogConfig': {'Type': '', 'Config': {}}}, networking_config={'EndpointsConfig': {'rootteststorages3queue_gw0_default': {'Aliases': ['minio1'], 'IPAMConfig': {}}}}) (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_container -> {'Id': 'dd458dce6abd41556f8b47dcab9a3965ec967f9cd91df20c365d9aa3e2016541', (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Warnings': []} (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_container -> {'Id': '3b0ace3d2b25df8abce5b8f56acee6953b512b6c7631f42247ccfd5a92fde157', (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Warnings': []} (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container <- ('3b0ace3d2b25df8abce5b8f56acee6953b512b6c7631f42247ccfd5a92fde157') (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container <- ('dd458dce6abd41556f8b47dcab9a3965ec967f9cd91df20c365d9aa3e2016541') (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container -> {'AppArmorProfile': '', (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Args': [], (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Config': {'AttachStderr': False, (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Cmd': ['python3'], (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Entrypoint': None, (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Env': ['PATH=/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin', (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'GPG_KEY=7169605F62C751356D054A26A821E680E5FA6305', (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker disconnect_container_from_network <- ('3b0ace3d2b25df8abce5b8f56acee6953b512b6c7631f42247ccfd5a92fde157', 'rootteststorages3queue_gw0_default') (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container -> {'AppArmorProfile': '', (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Args': ['server', (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: '--console-address', (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: '127.0.0.1:19001', (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: '--address', (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: ':9001', (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: '--certs-dir', (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: '/certs', (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: '/data1-1'], (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr: 'Config': {'AttachStderr': False, (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker disconnect_container_from_network <- ('dd458dce6abd41556f8b47dcab9a3965ec967f9cd91df20c365d9aa3e2016541', 'rootteststorages3queue_gw0_default') (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker disconnect_container_from_network -> None (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker connect_container_to_network <- ('dd458dce6abd41556f8b47dcab9a3965ec967f9cd91df20c365d9aa3e2016541', 'rootteststorages3queue_gw0_default', aliases=['minio1', 'dd458dce6abd'], ipv4_address=None, ipv6_address=None, links=[], link_local_ips=None) (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker disconnect_container_from_network -> None (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker connect_container_to_network <- ('3b0ace3d2b25df8abce5b8f56acee6953b512b6c7631f42247ccfd5a92fde157', 'rootteststorages3queue_gw0_default', aliases=['3b0ace3d2b25', 'resolver'], ipv4_address=None, ipv6_address=None, links=[], link_local_ips=None) (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker connect_container_to_network -> None (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker start <- ('dd458dce6abd41556f8b47dcab9a3965ec967f9cd91df20c365d9aa3e2016541') (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker connect_container_to_network -> None (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker start <- ('3b0ace3d2b25df8abce5b8f56acee6953b512b6c7631f42247ccfd5a92fde157') (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker start -> None (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker start -> None (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.parallel.parallel_execute_iter: Finished processing: ServiceName(project='rootteststorages3queue_gw0', service='resolver', number=1) (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:Creating rootteststorages3queue_gw0_resolver_1 ... done (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.parallel.parallel_execute_iter: Finished processing: (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.parallel.parallel_execute_iter: Finished processing: ServiceName(project='rootteststorages3queue_gw0', service='minio1', number=1) (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:Creating rootteststorages3queue_gw0_minio1_1 ... done (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.parallel.parallel_execute_iter: Finished processing: (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:00 [ 408 ] INFO : Trying to connect to Minio... (cluster.py:3097, start) 2026-10-30 18:52:00 [ 408 ] DEBUG : get_instance_ip instance_name=minio1 (cluster.py:2137, get_instance_ip) 2026-10-30 18:52:00 [ 408 ] DEBUG : http://localhost:None "GET /v1.42/containers/rootteststorages3queue_gw0_minio1_1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-10-30 18:52:00 [ 408 ] DEBUG : get_instance_ip instance_name=proxy1 (cluster.py:2137, get_instance_ip) 2026-10-30 18:52:00 [ 408 ] DEBUG : http://localhost:None "GET /v1.42/containers/rootteststorages3queue_gw0_proxy1_1/json HTTP/1.1" 200 None (connectionpool.py:547, _make_request) 2026-10-30 18:52:00 [ 408 ] DEBUG : Starting new HTTP connection (1): 172.16.10.7:9001 (connectionpool.py:245, _new_conn) 2026-10-30 18:52:00 [ 408 ] DEBUG : http://172.16.10.7:9001 "GET / HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-10-30 18:52:00 [ 408 ] DEBUG : Connected to Minio. (cluster.py:2651, wait_minio_to_start) 2026-10-30 18:52:00 [ 408 ] DEBUG : http://172.16.10.7:9001 "GET /root?location= HTTP/1.1" 404 0 (connectionpool.py:547, _make_request) 2026-10-30 18:52:00 [ 408 ] DEBUG : http://172.16.10.7:9001 "PUT /root HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-10-30 18:52:00 [ 408 ] DEBUG : S3 bucket 'root' created (cluster.py:2666, wait_minio_to_start) 2026-10-30 18:52:00 [ 408 ] DEBUG : http://172.16.10.7:9001 "GET /root2?location= HTTP/1.1" 404 0 (connectionpool.py:547, _make_request) 2026-10-30 18:52:00 [ 408 ] DEBUG : http://172.16.10.7:9001 "PUT /root2 HTTP/1.1" 200 0 (connectionpool.py:547, _make_request) 2026-10-30 18:52:00 [ 408 ] DEBUG : S3 bucket 'root2' created (cluster.py:2666, wait_minio_to_start) 2026-10-30 18:52:00 [ 408 ] INFO : Trying to create Azurite instance by command docker-compose --env-file /ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/.env --project-name rootteststorages3queue_gw0 --file /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_azurite.yml --verbose up -d (cluster.py:3102, start) 2026-10-30 18:52:00 [ 408 ] DEBUG : Command:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/.env', '--project-name', 'rootteststorages3queue_gw0', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_azurite.yml', '--verbose', 'up', '-d'] (cluster.py:113, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.config.config.find: Using configuration files: /ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_azurite.yml (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.cli.docker_client.get_client: docker-compose version 1.29.2, build unknown (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:docker-py version: (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:CPython version: 3.10.12 (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:OpenSSL version: OpenSSL 3.0.2 15 Mar 2022 (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.cli.docker_client.get_client: Docker base_url: http+docker://localhost (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.cli.docker_client.get_client: Docker version: Platform={'Name': 'Docker Engine - Community'}, Components=[{'Name': 'Engine', 'Version': '23.0.6', 'Details': {'ApiVersion': '1.42', 'Arch': 'arm64', 'BuildTime': '2023-05-05T21:18:33.000000000+00:00', 'Experimental': 'false', 'GitCommit': '9dbdbd4', 'GoVersion': 'go1.19.9', 'KernelVersion': '5.15.0-198-generic', 'MinAPIVersion': '1.12', 'Os': 'linux'}}, {'Name': 'containerd', 'Version': 'v2.2.6', 'Details': {'GitCommit': '11ce9d5f3c68c941867e82890e93e815c1304f1b'}}, {'Name': 'runc', 'Version': '1.3.6', 'Details': {'GitCommit': 'v1.3.6-0-g491b69ba'}}, {'Name': 'docker-init', 'Version': '0.19.0', 'Details': {'GitCommit': 'de40ad0'}}], Version=23.0.6, ApiVersion=1.42, MinAPIVersion=1.12, GitCommit=9dbdbd4, GoVersion=go1.19.9, Os=linux, Arch=arm64, KernelVersion=5.15.0-198-generic, BuildTime=2023-05-05T21:18:33.000000000+00:00 (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_network <- ('rootteststorages3queuegw0_default') (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_volume <- ('rootteststorages3queuegw0_data1-1') (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker info <- () (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker info -> {'Architecture': 'aarch64', (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'BridgeNfIp6tables': True, (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'BridgeNfIptables': True, (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'CPUSet': True, (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'CPUShares': True, (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'CgroupDriver': 'cgroupfs', (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'CgroupVersion': '2', (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'ContainerdCommit': {'Expected': '11ce9d5f3c68c941867e82890e93e815c1304f1b', (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'ID': '11ce9d5f3c68c941867e82890e93e815c1304f1b'}, (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'Containers': 41, (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_network <- ('rootteststorages3queue_gw0_default') (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_network -> {'Attachable': True, (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'ConfigFrom': {'Network': ''}, (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'ConfigOnly': False, (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'Containers': {'1f40c6eafa1401644cbd49e44d1c7cd8b5c7f792429edd711392c265c4b2def4': {'EndpointID': 'eb17be299b0aaa6738af7d25794dde23266cb63850519c573a8bfced50065494', (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'IPv4Address': '172.16.10.3/24', (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'IPv6Address': '', (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'MacAddress': '02:42:ac:10:0a:03', (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'Name': 'rootteststorages3queue_gw0_zoo3_1'}, (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: '3abd8efabb519533a4cc04dbe8e1d41a098b78e994a45e4f9022dfd90d231806': {'EndpointID': '74129e2d62ac300658e125fdcb26ff54a349b46e4d241d4b6f9ae597a2743955', (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'IPv4Address': '172.16.10.2/24', (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_volume <- ('rootteststorages3queue_gw0_data1-1') (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_volume -> {'CreatedAt': '2026-10-30T18:51:56Z', (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'Driver': 'local', (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'Labels': {'com.docker.compose.project': 'rootteststorages3queue_gw0', (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'com.docker.compose.version': '1.29.2', (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'com.docker.compose.volume': 'data1-1'}, (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'Mountpoint': '/var/lib/docker/volumes/rootteststorages3queue_gw0_data1-1/_data', (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'Name': 'rootteststorages3queue_gw0_data1-1', (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'Options': None, (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'Scope': 'local'} (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_volume <- ('rootteststorages3queue_gw0_data1-1') (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_volume -> {'CreatedAt': '2026-10-30T18:51:56Z', (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'Driver': 'local', (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'Labels': {'com.docker.compose.project': 'rootteststorages3queue_gw0', (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'com.docker.compose.version': '1.29.2', (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'com.docker.compose.volume': 'data1-1'}, (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'Mountpoint': '/var/lib/docker/volumes/rootteststorages3queue_gw0_data1-1/_data', (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'Name': 'rootteststorages3queue_gw0_data1-1', (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'Options': None, (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'Scope': 'local'} (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=False, filters={'label': ['com.docker.compose.project=rootteststorages3queue_gw0', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 7 items) (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue_gw0', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 7 items) (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container <- ('dd458dce6abd41556f8b47dcab9a3965ec967f9cd91df20c365d9aa3e2016541') (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container -> {'AppArmorProfile': '', (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'Args': ['server', (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: '--console-address', (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: '127.0.0.1:19001', (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: '--address', (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: ':9001', (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: '--certs-dir', (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: '/certs', (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: '/data1-1'], (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'Config': {'AttachStderr': False, (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container <- ('55184b41c92593eda8d504918421ddd7632b28c3a8ad2ddda9dd6373681d1194') (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container -> {'AppArmorProfile': '', (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'Args': ['/run.sh'], (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'Config': {'AttachStderr': False, (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'Cmd': ['/run.sh'], (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'Entrypoint': ['/docker-entrypoint.sh'], (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'Env': ['PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin', (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'NGINX_VERSION=1.27.5', (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container <- ('70924df5ac9ac27415657dd1665d0e0d9caeb6741e354c44a13b490d29d373d9') (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container -> {'AppArmorProfile': '', (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'Args': ['/run.sh'], (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'Config': {'AttachStderr': False, (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'Cmd': ['/run.sh'], (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'Entrypoint': ['/docker-entrypoint.sh'], (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'Env': ['PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin', (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'NGINX_VERSION=1.27.5', (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container <- ('3abd8efabb519533a4cc04dbe8e1d41a098b78e994a45e4f9022dfd90d231806') (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container -> {'AppArmorProfile': '', (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'Args': ['keeper', (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: '--config=/etc/clickhouse-keeper/keeper_config1.xml', (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: '--log-file=/var/log/clickhouse-keeper/clickhouse-keeper.log', (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: '--errorlog-file=/var/log/clickhouse-keeper/clickhouse-keeper.err.log'], (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'Config': {'AttachStderr': False, (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'Cmd': None, (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container <- ('3b0ace3d2b25df8abce5b8f56acee6953b512b6c7631f42247ccfd5a92fde157') (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container -> {'AppArmorProfile': '', (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'Args': [], (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'Config': {'AttachStderr': False, (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'Cmd': ['python3'], (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'Entrypoint': None, (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'Env': ['PATH=/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin', (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'GPG_KEY=7169605F62C751356D054A26A821E680E5FA6305', (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container <- ('71024ce75a4999e2cbf31df30572e641bb088f736f7fe01259a50b8796c1c29c') (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container -> {'AppArmorProfile': '', (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'Args': ['keeper', (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: '--config=/etc/clickhouse-keeper/keeper_config2.xml', (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: '--log-file=/var/log/clickhouse-keeper/clickhouse-keeper.log', (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: '--errorlog-file=/var/log/clickhouse-keeper/clickhouse-keeper.err.log'], (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'Config': {'AttachStderr': False, (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'Cmd': None, (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container <- ('1f40c6eafa1401644cbd49e44d1c7cd8b5c7f792429edd711392c265c4b2def4') (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container -> {'AppArmorProfile': '', (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'Args': ['keeper', (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: '--config=/etc/clickhouse-keeper/keeper_config3.xml', (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: '--log-file=/var/log/clickhouse-keeper/clickhouse-keeper.log', (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: '--errorlog-file=/var/log/clickhouse-keeper/clickhouse-keeper.err.log'], (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'Config': {'AttachStderr': False, (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'Cmd': None, (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'Domainname': '', (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.project.find_orphan_containers: Found orphan containers (rootteststorages3queue_gw0_minio1_1, rootteststorages3queue_gw0_proxy2_1, rootteststorages3queue_gw0_proxy1_1, rootteststorages3queue_gw0_zoo1_1, rootteststorages3queue_gw0_resolver_1, rootteststorages3queue_gw0_zoo2_1, rootteststorages3queue_gw0_zoo3_1) for this project. If you removed or renamed this service in your compose file, you can run this command with the --remove-orphans flag to clean it up. (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue_gw0', 'com.docker.compose.service=azurite1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queuegw0', 'com.docker.compose.service=azurite1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/azurite:3.35.0') (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'amd64', (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'Author': '', (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'AttachStderr': False, (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'Cmd': ['azurite', (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: '-l', (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: '/data', (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue_gw0', 'com.docker.compose.service=azurite1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queuegw0', 'com.docker.compose.service=azurite1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: {} (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Starting producer thread for (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queue_gw0', 'com.docker.compose.service=azurite1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers <- (all=True, filters={'label': ['com.docker.compose.project=rootteststorages3queuegw0', 'com.docker.compose.service=azurite1', 'com.docker.compose.oneoff=False']}) (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker containers -> (list with 0 items) (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:Creating rootteststorages3queue_gw0_azurite1_1 ... (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: {ServiceName(project='rootteststorages3queue_gw0', service='azurite1', number=1)} (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Starting producer thread for ServiceName(project='rootteststorages3queue_gw0', service='azurite1', number=1) (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/azurite:3.35.0') (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'amd64', (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'Author': '', (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'AttachStderr': False, (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'Cmd': ['azurite', (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: '-l', (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: '/data', (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image <- ('altinityinfra/azurite:3.35.0') (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_image -> {'Architecture': 'amd64', (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'Author': '', (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'Comment': 'buildkit.dockerfile.v0', (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'Config': {'ArgsEscaped': True, (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'AttachStderr': False, (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'AttachStdout': False, (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'Cmd': ['azurite', (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: '-l', (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: '/data', (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.service.build_container_labels: Added config hash: 623a37a7b8c9e2c64d43745b96cc6ab971f719ede3d07cab648e26068e75d3de (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_host_config <- (links=[], port_bindings={'30000/tcp': ['30000']}, binds=['rootteststorages3queue_gw0_data1-1:/data1:rw'], volumes_from=[], privileged=False, network_mode='rootteststorages3queue_gw0_default', devices=None, device_requests=None, dns=None, dns_opt=None, dns_search=None, restart_policy=None, runtime=None, cap_add=None, cap_drop=None, mem_limit=None, mem_reservation=None, memswap_limit=None, ulimits=None, log_config={'Type': '', 'Config': {}}, extra_hosts=None, read_only=None, pid_mode=None, security_opt=None, ipc_mode=None, cgroup_parent=None, cpu_quota=None, shm_size=None, sysctls=None, pids_limit=None, tmpfs=None, oom_kill_disable=None, oom_score_adj=None, mem_swappiness=None, group_add=None, userns_mode=None, init=None, init_path=None, isolation=None, cpu_count=None, cpu_percent=None, nano_cpus=None, volume_driver=None, cpuset_cpus=None, cpu_shares=None, storage_opt=None, blkio_weight=None, blkio_weight_device=None, device_read_bps=None, device_read_iops=None, device_write_bps=None, device_write_iops=None, mounts=None, device_cgroup_rules=None, cpu_period=None, cpu_rt_period=None, cpu_rt_runtime=None) (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_host_config -> {'Binds': ['rootteststorages3queue_gw0_data1-1:/data1:rw'], (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'Links': [], (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'LogConfig': {'Config': {}, 'Type': ''}, (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'NetworkMode': 'rootteststorages3queue_gw0_default', (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'PortBindings': {'30000/tcp': [{'HostIp': '', 'HostPort': '30000'}]}, (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'VolumesFrom': []} (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_container <- (command='azurite-blob --blobHost 0.0.0.0 --blobPort 30000 --debug /azurite_log', image='altinityinfra/azurite:3.35.0', ports=[('30000', 'tcp')], volumes={'/data1': {}}, name='rootteststorages3queue_gw0_azurite1_1', detach=True, environment=[], labels={'com.docker.compose.project': 'rootteststorages3queue_gw0', 'com.docker.compose.service': 'azurite1', 'com.docker.compose.oneoff': 'False', 'com.docker.compose.project.working_dir': '/ClickHouse/tests/integration/compose', 'com.docker.compose.project.config_files': '/ClickHouse/tests/integration/compose/docker_compose_azurite.yml', 'com.docker.compose.project.environment_file': '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/.env', 'com.docker.compose.container-number': '1', 'com.docker.compose.version': '1.29.2', 'com.docker.compose.config-hash': '623a37a7b8c9e2c64d43745b96cc6ab971f719ede3d07cab648e26068e75d3de'}, host_config={'NetworkMode': 'rootteststorages3queue_gw0_default', 'VolumesFrom': [], 'Binds': ['rootteststorages3queue_gw0_data1-1:/data1:rw'], 'PortBindings': {'30000/tcp': [{'HostIp': '', 'HostPort': '30000'}]}, 'Links': [], 'LogConfig': {'Type': '', 'Config': {}}}, networking_config={'EndpointsConfig': {'rootteststorages3queue_gw0_default': {'Aliases': ['azurite1'], 'IPAMConfig': {}}}}) (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker create_container -> {'Id': 'ff76105662fbbef2bba2cbf5f4ccc6ede8461d0e4e4a2e719002988f9ff8ea01', (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'Warnings': ["The requested image's platform (linux/amd64) does not match the " (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'detected host platform (linux/arm64/v8) and no specific ' (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'platform was requested']} (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container <- ('ff76105662fbbef2bba2cbf5f4ccc6ede8461d0e4e4a2e719002988f9ff8ea01') (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker inspect_container -> {'AppArmorProfile': '', (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'Args': ['azurite-blob', (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: '--blobHost', (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: '0.0.0.0', (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: '--blobPort', (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: '30000', (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: '--debug', (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: '/azurite_log'], (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'Config': {'AttachStderr': False, (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr: 'AttachStdin': False, (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:... (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker disconnect_container_from_network <- ('ff76105662fbbef2bba2cbf5f4ccc6ede8461d0e4e4a2e719002988f9ff8ea01', 'rootteststorages3queue_gw0_default') (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker disconnect_container_from_network -> None (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker connect_container_to_network <- ('ff76105662fbbef2bba2cbf5f4ccc6ede8461d0e4e4a2e719002988f9ff8ea01', 'rootteststorages3queue_gw0_default', aliases=['azurite1', 'ff76105662fb'], ipv4_address=None, ipv6_address=None, links=[], link_local_ips=None) (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker connect_container_to_network -> None (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker start <- ('ff76105662fbbef2bba2cbf5f4ccc6ede8461d0e4e4a2e719002988f9ff8ea01') (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.cli.verbose_proxy.proxy_callable: docker start -> None (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.parallel.parallel_execute_iter: Finished processing: ServiceName(project='rootteststorages3queue_gw0', service='azurite1', number=1) (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:Creating rootteststorages3queue_gw0_azurite1_1 ... done (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.parallel.parallel_execute_iter: Finished processing: (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] DEBUG : Stderr:compose.parallel.feed_queue: Pending: set() (cluster.py:123, run_and_check) 2026-10-30 18:52:07 [ 408 ] INFO : Trying to connect to Azurite (cluster.py:3118, start) 2026-10-30 18:52:08 [ 408 ] INFO : Request URL: 'http://127.0.0.1:30000/devstoreaccount1/?restype=REDACTED&comp=REDACTED' Request method: 'GET' Request headers: 'x-ms-version': 'REDACTED' 'Accept': 'application/xml' 'User-Agent': 'azsdk-python-storage-blob/12.19.1 Python/3.10.12 (Linux-5.15.0-198-generic-aarch64-with-glibc2.35)' 'x-ms-date': 'REDACTED' 'x-ms-client-request-id': '0224f7d6-d493-11f1-b83f-7ae1f1890d30' 'Authorization': 'REDACTED' No body was attached to the request (_universal.py:513, on_request) 2026-10-30 18:52:08 [ 408 ] DEBUG : Starting new HTTP connection (1): 127.0.0.1:30000 (connectionpool.py:245, _new_conn) 2026-10-30 18:52:28 [ 408 ] INFO : Request URL: 'http://127.0.0.1:30000/devstoreaccount1/?restype=REDACTED&comp=REDACTED' Request method: 'GET' Request headers: 'x-ms-version': 'REDACTED' 'Accept': 'application/xml' 'User-Agent': 'azsdk-python-storage-blob/12.19.1 Python/3.10.12 (Linux-5.15.0-198-generic-aarch64-with-glibc2.35)' 'x-ms-date': 'REDACTED' 'x-ms-client-request-id': '0e300dcc-d493-11f1-b83f-7ae1f1890d30' 'Authorization': 'REDACTED' No body was attached to the request (_universal.py:513, on_request) 2026-10-30 18:52:28 [ 408 ] DEBUG : Starting new HTTP connection (2): 127.0.0.1:30000 (connectionpool.py:245, _new_conn) 2026-10-30 18:52:55 [ 408 ] INFO : Request URL: 'http://127.0.0.1:30000/devstoreaccount1/?restype=REDACTED&comp=REDACTED' Request method: 'GET' Request headers: 'x-ms-version': 'REDACTED' 'Accept': 'application/xml' 'User-Agent': 'azsdk-python-storage-blob/12.19.1 Python/3.10.12 (Linux-5.15.0-198-generic-aarch64-with-glibc2.35)' 'x-ms-date': 'REDACTED' 'x-ms-client-request-id': '1e2bd59e-d493-11f1-b83f-7ae1f1890d30' 'Authorization': 'REDACTED' No body was attached to the request (_universal.py:513, on_request) 2026-10-30 18:52:55 [ 408 ] DEBUG : Starting new HTTP connection (3): 127.0.0.1:30000 (connectionpool.py:245, _new_conn) 2026-10-30 18:53:40 [ 408 ] INFO : Request URL: 'http://127.0.0.1:30000/devstoreaccount1/?restype=REDACTED&comp=REDACTED' Request method: 'GET' Request headers: 'x-ms-version': 'REDACTED' 'Accept': 'application/xml' 'User-Agent': 'azsdk-python-storage-blob/12.19.1 Python/3.10.12 (Linux-5.15.0-198-generic-aarch64-with-glibc2.35)' 'x-ms-date': 'REDACTED' 'x-ms-client-request-id': '38ffc5a6-d493-11f1-b83f-7ae1f1890d30' 'Authorization': 'REDACTED' No body was attached to the request (_universal.py:513, on_request) 2026-10-30 18:53:40 [ 408 ] DEBUG : Starting new HTTP connection (4): 127.0.0.1:30000 (connectionpool.py:245, _new_conn) 2026-10-30 18:53:40 [ 408 ] DEBUG : Can't connect to Azurite: ('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer')) (cluster.py:2703, wait_azurite_to_start) 2026-10-30 18:53:41 [ 408 ] INFO : Request URL: 'http://127.0.0.1:30000/devstoreaccount1/?restype=REDACTED&comp=REDACTED' Request method: 'GET' Request headers: 'x-ms-version': 'REDACTED' 'Accept': 'application/xml' 'User-Agent': 'azsdk-python-storage-blob/12.19.1 Python/3.10.12 (Linux-5.15.0-198-generic-aarch64-with-glibc2.35)' 'x-ms-date': 'REDACTED' 'x-ms-client-request-id': '3999a644-d493-11f1-b83f-7ae1f1890d30' 'Authorization': 'REDACTED' No body was attached to the request (_universal.py:513, on_request) 2026-10-30 18:53:41 [ 408 ] DEBUG : Starting new HTTP connection (1): 127.0.0.1:30000 (connectionpool.py:245, _new_conn) 2026-10-30 18:54:01 [ 408 ] INFO : Request URL: 'http://127.0.0.1:30000/devstoreaccount1/?restype=REDACTED&comp=REDACTED' Request method: 'GET' Request headers: 'x-ms-version': 'REDACTED' 'Accept': 'application/xml' 'User-Agent': 'azsdk-python-storage-blob/12.19.1 Python/3.10.12 (Linux-5.15.0-198-generic-aarch64-with-glibc2.35)' 'x-ms-date': 'REDACTED' 'x-ms-client-request-id': '45a9e1a6-d493-11f1-b83f-7ae1f1890d30' 'Authorization': 'REDACTED' No body was attached to the request (_universal.py:513, on_request) 2026-10-30 18:54:01 [ 408 ] DEBUG : Starting new HTTP connection (2): 127.0.0.1:30000 (connectionpool.py:245, _new_conn) 2026-10-30 18:54:23 [ 408 ] INFO : Request URL: 'http://127.0.0.1:30000/devstoreaccount1/?restype=REDACTED&comp=REDACTED' Request method: 'GET' Request headers: 'x-ms-version': 'REDACTED' 'Accept': 'application/xml' 'User-Agent': 'azsdk-python-storage-blob/12.19.1 Python/3.10.12 (Linux-5.15.0-198-generic-aarch64-with-glibc2.35)' 'x-ms-date': 'REDACTED' 'x-ms-client-request-id': '52cdfc78-d493-11f1-b83f-7ae1f1890d30' 'Authorization': 'REDACTED' No body was attached to the request (_universal.py:513, on_request) 2026-10-30 18:54:23 [ 408 ] DEBUG : Starting new HTTP connection (3): 127.0.0.1:30000 (connectionpool.py:245, _new_conn) 2026-10-30 18:55:07 [ 408 ] INFO : Request URL: 'http://127.0.0.1:30000/devstoreaccount1/?restype=REDACTED&comp=REDACTED' Request method: 'GET' Request headers: 'x-ms-version': 'REDACTED' 'Accept': 'application/xml' 'User-Agent': 'azsdk-python-storage-blob/12.19.1 Python/3.10.12 (Linux-5.15.0-198-generic-aarch64-with-glibc2.35)' 'x-ms-date': 'REDACTED' 'x-ms-client-request-id': '6ccb94fa-d493-11f1-b83f-7ae1f1890d30' 'Authorization': 'REDACTED' No body was attached to the request (_universal.py:513, on_request) 2026-10-30 18:55:07 [ 408 ] DEBUG : Starting new HTTP connection (4): 127.0.0.1:30000 (connectionpool.py:245, _new_conn) 2026-10-30 18:55:07 [ 408 ] DEBUG : Can't connect to Azurite: ('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer')) (cluster.py:2703, wait_azurite_to_start) 2026-10-30 18:55:08 [ 408 ] INFO : Request URL: 'http://127.0.0.1:30000/devstoreaccount1/?restype=REDACTED&comp=REDACTED' Request method: 'GET' Request headers: 'x-ms-version': 'REDACTED' 'Accept': 'application/xml' 'User-Agent': 'azsdk-python-storage-blob/12.19.1 Python/3.10.12 (Linux-5.15.0-198-generic-aarch64-with-glibc2.35)' 'x-ms-date': 'REDACTED' 'x-ms-client-request-id': '6d65c34a-d493-11f1-b83f-7ae1f1890d30' 'Authorization': 'REDACTED' No body was attached to the request (_universal.py:513, on_request) 2026-10-30 18:55:08 [ 408 ] DEBUG : Starting new HTTP connection (1): 127.0.0.1:30000 (connectionpool.py:245, _new_conn) 2026-10-30 18:55:26 [ 408 ] INFO : Request URL: 'http://127.0.0.1:30000/devstoreaccount1/?restype=REDACTED&comp=REDACTED' Request method: 'GET' Request headers: 'x-ms-version': 'REDACTED' 'Accept': 'application/xml' 'User-Agent': 'azsdk-python-storage-blob/12.19.1 Python/3.10.12 (Linux-5.15.0-198-generic-aarch64-with-glibc2.35)' 'x-ms-date': 'REDACTED' 'x-ms-client-request-id': '77f0e09c-d493-11f1-b83f-7ae1f1890d30' 'Authorization': 'REDACTED' No body was attached to the request (_universal.py:513, on_request) 2026-10-30 18:55:26 [ 408 ] DEBUG : Starting new HTTP connection (2): 127.0.0.1:30000 (connectionpool.py:245, _new_conn) 2026-10-30 18:55:50 [ 408 ] INFO : Request URL: 'http://127.0.0.1:30000/devstoreaccount1/?restype=REDACTED&comp=REDACTED' Request method: 'GET' Request headers: 'x-ms-version': 'REDACTED' 'Accept': 'application/xml' 'User-Agent': 'azsdk-python-storage-blob/12.19.1 Python/3.10.12 (Linux-5.15.0-198-generic-aarch64-with-glibc2.35)' 'x-ms-date': 'REDACTED' 'x-ms-client-request-id': '868da892-d493-11f1-b83f-7ae1f1890d30' 'Authorization': 'REDACTED' No body was attached to the request (_universal.py:513, on_request) 2026-10-30 18:55:50 [ 408 ] DEBUG : Starting new HTTP connection (3): 127.0.0.1:30000 (connectionpool.py:245, _new_conn) 2026-10-30 18:56:34 [ 408 ] INFO : Request URL: 'http://127.0.0.1:30000/devstoreaccount1/?restype=REDACTED&comp=REDACTED' Request method: 'GET' Request headers: 'x-ms-version': 'REDACTED' 'Accept': 'application/xml' 'User-Agent': 'azsdk-python-storage-blob/12.19.1 Python/3.10.12 (Linux-5.15.0-198-generic-aarch64-with-glibc2.35)' 'x-ms-date': 'REDACTED' 'x-ms-client-request-id': 'a07cfdf2-d493-11f1-b83f-7ae1f1890d30' 'Authorization': 'REDACTED' No body was attached to the request (_universal.py:513, on_request) 2026-10-30 18:56:34 [ 408 ] DEBUG : Starting new HTTP connection (4): 127.0.0.1:30000 (connectionpool.py:245, _new_conn) 2026-10-30 18:56:34 [ 408 ] DEBUG : Can't connect to Azurite: ('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer')) (cluster.py:2703, wait_azurite_to_start) 2026-10-30 18:56:35 [ 408 ] DEBUG : Failed to start cluster: (cluster.py:3178, start) 2026-10-30 18:56:35 [ 408 ] DEBUG : Can't wait Azurite to start (cluster.py:3179, start) 2026-10-30 18:56:35 [ 408 ] DEBUG : None (cluster.py:3180, start) 2026-10-30 18:56:36 [ 408 ] DEBUG : Command:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/.env', '--project-name', 'rootteststorages3queue_gw0', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/instance/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_azurite.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/instance2/docker-compose.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/old_instance/docker-compose.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/instance_too_many_parts/docker-compose.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/instance_24.5/docker-compose.yml', 'stop', '--timeout', '20'] (cluster.py:113, run_and_check) 2026-10-30 18:56:58 [ 408 ] DEBUG : Stderr:Stopping rootteststorages3queue_gw0_azurite1_1 ... (cluster.py:123, run_and_check) 2026-10-30 18:56:58 [ 408 ] DEBUG : Stderr:Stopping rootteststorages3queue_gw0_minio1_1 ... (cluster.py:123, run_and_check) 2026-10-30 18:56:58 [ 408 ] DEBUG : Stderr:Stopping rootteststorages3queue_gw0_resolver_1 ... (cluster.py:123, run_and_check) 2026-10-30 18:56:58 [ 408 ] DEBUG : Stderr:Stopping rootteststorages3queue_gw0_proxy1_1 ... (cluster.py:123, run_and_check) 2026-10-30 18:56:58 [ 408 ] DEBUG : Stderr:Stopping rootteststorages3queue_gw0_proxy2_1 ... (cluster.py:123, run_and_check) 2026-10-30 18:56:58 [ 408 ] DEBUG : Stderr:Stopping rootteststorages3queue_gw0_zoo1_1 ... (cluster.py:123, run_and_check) 2026-10-30 18:56:58 [ 408 ] DEBUG : Stderr:Stopping rootteststorages3queue_gw0_zoo3_1 ... (cluster.py:123, run_and_check) 2026-10-30 18:56:58 [ 408 ] DEBUG : Stderr:Stopping rootteststorages3queue_gw0_zoo2_1 ... (cluster.py:123, run_and_check) 2026-10-30 18:56:58 [ 408 ] DEBUG : Stderr:Stopping rootteststorages3queue_gw0_minio1_1 ... done (cluster.py:123, run_and_check) 2026-10-30 18:56:58 [ 408 ] DEBUG : Stderr:Stopping rootteststorages3queue_gw0_azurite1_1 ... done (cluster.py:123, run_and_check) 2026-10-30 18:56:58 [ 408 ] DEBUG : Stderr:Stopping rootteststorages3queue_gw0_zoo2_1 ... done (cluster.py:123, run_and_check) 2026-10-30 18:56:58 [ 408 ] DEBUG : Stderr:Stopping rootteststorages3queue_gw0_zoo1_1 ... done (cluster.py:123, run_and_check) 2026-10-30 18:56:58 [ 408 ] DEBUG : Stderr:Stopping rootteststorages3queue_gw0_zoo3_1 ... done (cluster.py:123, run_and_check) 2026-10-30 18:56:58 [ 408 ] DEBUG : Stderr:Stopping rootteststorages3queue_gw0_resolver_1 ... done (cluster.py:123, run_and_check) 2026-10-30 18:56:58 [ 408 ] DEBUG : Stderr:Stopping rootteststorages3queue_gw0_proxy1_1 ... done (cluster.py:123, run_and_check) 2026-10-30 18:56:58 [ 408 ] DEBUG : Stderr:Stopping rootteststorages3queue_gw0_proxy2_1 ... done (cluster.py:123, run_and_check) 2026-10-30 18:56:58 [ 408 ] DEBUG : Command:['bash', '-c', '[ -f /ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/instance/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/instance/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true'] (cluster.py:113, run_and_check) 2026-10-30 18:56:58 [ 408 ] DEBUG : Command:['bash', '-c', '[ -f /ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/instance2/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/instance2/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true'] (cluster.py:113, run_and_check) 2026-10-30 18:56:58 [ 408 ] DEBUG : Command:['bash', '-c', '[ -f /ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/old_instance/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/old_instance/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true'] (cluster.py:113, run_and_check) 2026-10-30 18:56:58 [ 408 ] DEBUG : Command:['bash', '-c', '[ -f /ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/instance_too_many_parts/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/instance_too_many_parts/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true'] (cluster.py:113, run_and_check) 2026-10-30 18:56:58 [ 408 ] DEBUG : Command:['bash', '-c', '[ -f /ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/instance_24.5/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/instance_24.5/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true'] (cluster.py:113, run_and_check) 2026-10-30 18:56:58 [ 408 ] DEBUG : Command:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/.env', '--project-name', 'rootteststorages3queue_gw0', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/instance/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_azurite.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/instance2/docker-compose.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/old_instance/docker-compose.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/instance_too_many_parts/docker-compose.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/instance_24.5/docker-compose.yml', 'down', '--volumes'] (cluster.py:113, run_and_check) 2026-10-30 18:57:00 [ 408 ] DEBUG : Stderr:Removing rootteststorages3queue_gw0_azurite1_1 ... (cluster.py:123, run_and_check) 2026-10-30 18:57:00 [ 408 ] DEBUG : Stderr:Removing rootteststorages3queue_gw0_minio1_1 ... (cluster.py:123, run_and_check) 2026-10-30 18:57:00 [ 408 ] DEBUG : Stderr:Removing rootteststorages3queue_gw0_resolver_1 ... (cluster.py:123, run_and_check) 2026-10-30 18:57:00 [ 408 ] DEBUG : Stderr:Removing rootteststorages3queue_gw0_proxy1_1 ... (cluster.py:123, run_and_check) 2026-10-30 18:57:00 [ 408 ] DEBUG : Stderr:Removing rootteststorages3queue_gw0_proxy2_1 ... (cluster.py:123, run_and_check) 2026-10-30 18:57:00 [ 408 ] DEBUG : Stderr:Removing rootteststorages3queue_gw0_zoo1_1 ... (cluster.py:123, run_and_check) 2026-10-30 18:57:00 [ 408 ] DEBUG : Stderr:Removing rootteststorages3queue_gw0_zoo3_1 ... (cluster.py:123, run_and_check) 2026-10-30 18:57:00 [ 408 ] DEBUG : Stderr:Removing rootteststorages3queue_gw0_zoo2_1 ... (cluster.py:123, run_and_check) 2026-10-30 18:57:00 [ 408 ] DEBUG : Stderr:Removing rootteststorages3queue_gw0_proxy2_1 ... done (cluster.py:123, run_and_check) 2026-10-30 18:57:00 [ 408 ] DEBUG : Stderr:Removing rootteststorages3queue_gw0_zoo2_1 ... done (cluster.py:123, run_and_check) 2026-10-30 18:57:00 [ 408 ] DEBUG : Stderr:Removing rootteststorages3queue_gw0_zoo1_1 ... done (cluster.py:123, run_and_check) 2026-10-30 18:57:00 [ 408 ] DEBUG : Stderr:Removing rootteststorages3queue_gw0_resolver_1 ... done (cluster.py:123, run_and_check) 2026-10-30 18:57:00 [ 408 ] DEBUG : Stderr:Removing rootteststorages3queue_gw0_proxy1_1 ... done (cluster.py:123, run_and_check) 2026-10-30 18:57:00 [ 408 ] DEBUG : Stderr:Removing rootteststorages3queue_gw0_zoo3_1 ... done (cluster.py:123, run_and_check) 2026-10-30 18:57:00 [ 408 ] DEBUG : Stderr:Removing rootteststorages3queue_gw0_azurite1_1 ... done (cluster.py:123, run_and_check) 2026-10-30 18:57:00 [ 408 ] DEBUG : Stderr:Removing rootteststorages3queue_gw0_minio1_1 ... done (cluster.py:123, run_and_check) 2026-10-30 18:57:00 [ 408 ] DEBUG : Stderr:Removing network rootteststorages3queue_gw0_default (cluster.py:123, run_and_check) 2026-10-30 18:57:00 [ 408 ] DEBUG : Stderr:Removing volume rootteststorages3queue_gw0_data1-1 (cluster.py:123, run_and_check) 2026-10-30 18:57:00 [ 408 ] DEBUG : Cleanup called (cluster.py:876, cleanup) 2026-10-30 18:57:00 [ 408 ] DEBUG : Docker networks for project rootteststorages3queue_gw0 are NETWORK ID NAME DRIVER SCOPE (cluster.py:855, print_all_docker_pieces) 2026-10-30 18:57:00 [ 408 ] DEBUG : Docker containers for project rootteststorages3queue_gw0 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES (cluster.py:863, print_all_docker_pieces) 2026-10-30 18:57:00 [ 408 ] DEBUG : Docker volumes for project rootteststorages3queue_gw0 are DRIVER VOLUME NAME (cluster.py:871, print_all_docker_pieces) 2026-10-30 18:57:00 [ 408 ] DEBUG : Command:docker container list --all --filter name='^/rootteststorages3queue_gw0_.*_1$' --format '{{.ID}}:{{.Names}}' (cluster.py:113, run_and_check) 2026-10-30 18:57:00 [ 408 ] DEBUG : Unstopped containers: {} (cluster.py:890, cleanup) 2026-10-30 18:57:00 [ 408 ] DEBUG : No running containers for project: rootteststorages3queue_gw0 (cluster.py:904, cleanup) 2026-10-30 18:57:00 [ 408 ] DEBUG : Trying to prune unused networks... (cluster.py:910, cleanup) 2026-10-30 18:57:00 [ 408 ] DEBUG : Trying to prune unused volumes... (cluster.py:937, cleanup) 2026-10-30 18:57:00 [ 408 ] DEBUG : Command:['docker volume ls | wc -l'] (cluster.py:113, run_and_check) 2026-10-30 18:57:00 [ 408 ] DEBUG : Stdout:3 (cluster.py:121, run_and_check) 2026-10-30 18:57:01 [ 408 ] DEBUG : Command:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/.env', '--project-name', 'rootteststorages3queue_gw0', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/instance/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_azurite.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/instance2/docker-compose.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/old_instance/docker-compose.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/instance_too_many_parts/docker-compose.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/instance_24.5/docker-compose.yml', 'stop', '--timeout', '20'] (cluster.py:113, run_and_check) 2026-10-30 18:57:03 [ 408 ] DEBUG : Command:['bash', '-c', '[ -f /ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/instance/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/instance/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true'] (cluster.py:113, run_and_check) 2026-10-30 18:57:03 [ 408 ] DEBUG : Command:['bash', '-c', '[ -f /ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/instance2/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/instance2/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true'] (cluster.py:113, run_and_check) 2026-10-30 18:57:03 [ 408 ] DEBUG : Command:['bash', '-c', '[ -f /ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/old_instance/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/old_instance/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true'] (cluster.py:113, run_and_check) 2026-10-30 18:57:03 [ 408 ] DEBUG : Command:['bash', '-c', '[ -f /ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/instance_too_many_parts/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/instance_too_many_parts/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true'] (cluster.py:113, run_and_check) 2026-10-30 18:57:03 [ 408 ] DEBUG : Command:['bash', '-c', '[ -f /ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/instance_24.5/logs/stderr.log ] && zgrep -aH "==================" /ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/instance_24.5/logs/stderr.log* | ( [ -z "" ] && cat || grep -v "$" ) || true'] (cluster.py:113, run_and_check) 2026-10-30 18:57:03 [ 408 ] DEBUG : Command:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/.env', '--project-name', 'rootteststorages3queue_gw0', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/instance/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_keeper.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_azurite.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/instance2/docker-compose.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/old_instance/docker-compose.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/instance_too_many_parts/docker-compose.yml', '--file', '/ClickHouse/tests/integration/test_storage_s3_queue/_instances_0_gw0/instance_24.5/docker-compose.yml', 'down', '--volumes'] (cluster.py:113, run_and_check) 2026-10-30 18:57:04 [ 408 ] DEBUG : Stderr:Removing network rootteststorages3queue_gw0_default (cluster.py:123, run_and_check) 2026-10-30 18:57:04 [ 408 ] DEBUG : Stderr:Network rootteststorages3queue_gw0_default not found. (cluster.py:123, run_and_check) 2026-10-30 18:57:04 [ 408 ] DEBUG : Stderr:Removing volume rootteststorages3queue_gw0_data1-1 (cluster.py:123, run_and_check) 2026-10-30 18:57:04 [ 408 ] DEBUG : Stderr:Volume rootteststorages3queue_gw0_data1-1 not found. (cluster.py:123, run_and_check) 2026-10-30 18:57:04 [ 408 ] DEBUG : Cleanup called (cluster.py:876, cleanup) 2026-10-30 18:57:04 [ 408 ] DEBUG : Docker networks for project rootteststorages3queue_gw0 are NETWORK ID NAME DRIVER SCOPE (cluster.py:855, print_all_docker_pieces) 2026-10-30 18:57:04 [ 408 ] DEBUG : Docker containers for project rootteststorages3queue_gw0 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES (cluster.py:863, print_all_docker_pieces) 2026-10-30 18:57:04 [ 408 ] DEBUG : Docker volumes for project rootteststorages3queue_gw0 are DRIVER VOLUME NAME (cluster.py:871, print_all_docker_pieces) 2026-10-30 18:57:04 [ 408 ] DEBUG : Command:docker container list --all --filter name='^/rootteststorages3queue_gw0_.*_1$' --format '{{.ID}}:{{.Names}}' (cluster.py:113, run_and_check) 2026-10-30 18:57:04 [ 408 ] DEBUG : Unstopped containers: {} (cluster.py:890, cleanup) 2026-10-30 18:57:04 [ 408 ] DEBUG : No running containers for project: rootteststorages3queue_gw0 (cluster.py:904, cleanup) 2026-10-30 18:57:04 [ 408 ] DEBUG : Trying to prune unused networks... (cluster.py:910, cleanup) 2026-10-30 18:57:04 [ 408 ] DEBUG : Trying to prune unused volumes... (cluster.py:937, cleanup) 2026-10-30 18:57:04 [ 408 ] DEBUG : Command:['docker volume ls | wc -l'] (cluster.py:113, run_and_check) 2026-10-30 18:57:05 [ 408 ] DEBUG : Stdout:3 (cluster.py:121, run_and_check) _________ ERROR at setup of test_direct_select_multiple_files[ordered] _________ [gw0] linux -- Python 3.10.12 /usr/bin/python3 @pytest.fixture(scope="module") def started_cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "instance", user_configs=["configs/users.xml"], with_minio=True, with_azurite=True, with_zookeeper=True, main_configs=[ "configs/zookeeper.xml", "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "instance2", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "old_instance", user_configs=["configs/users_old.xml"], with_zookeeper=True, # NOTE (vnemkov) Can't use altinity/clickhouse here since 23.8 doesn't hve S3Queue (and associated settings yet) image="clickhouse/clickhouse-server", tag="23.12", stay_alive=True, with_installed_binary=True, use_old_analyzer=True, ) cluster.add_instance( "instance_too_many_parts", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", "configs/merge_tree.xml", ], stay_alive=True, ) cluster.add_instance( "instance_24.5", with_zookeeper=True, image="clickhouse/clickhouse-server", tag="24.5", stay_alive=True, user_configs=[ "configs/users.xml", ], with_installed_binary=True, use_old_analyzer=True, ) logging.info("Starting cluster...") > cluster.start() test_storage_s3_queue/test.py:144: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:3119: in start self.wait_azurite_to_start() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = timeout = 180 def wait_azurite_to_start(self, timeout=180): from azure.storage.blob import BlobServiceClient connection_string = ( f"DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;" f"AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;" f"BlobEndpoint=http://127.0.0.1:{self.env_variables['AZURITE_PORT']}/devstoreaccount1;" ) time.sleep(1) start = time.time() while time.time() - start < timeout: try: blob_service_client = BlobServiceClient.from_connection_string( connection_string ) logging.debug(blob_service_client.get_account_information()) self.blob_service_client = blob_service_client return except Exception as ex: logging.debug("Can't connect to Azurite: %s", str(ex)) time.sleep(1) > raise Exception("Can't wait Azurite to start") E Exception: Can't wait Azurite to start helpers/cluster.py:2706: Exception ________ ERROR at setup of test_direct_select_multiple_files[unordered] ________ [gw0] linux -- Python 3.10.12 /usr/bin/python3 @pytest.fixture(scope="module") def started_cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "instance", user_configs=["configs/users.xml"], with_minio=True, with_azurite=True, with_zookeeper=True, main_configs=[ "configs/zookeeper.xml", "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "instance2", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "old_instance", user_configs=["configs/users_old.xml"], with_zookeeper=True, # NOTE (vnemkov) Can't use altinity/clickhouse here since 23.8 doesn't hve S3Queue (and associated settings yet) image="clickhouse/clickhouse-server", tag="23.12", stay_alive=True, with_installed_binary=True, use_old_analyzer=True, ) cluster.add_instance( "instance_too_many_parts", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", "configs/merge_tree.xml", ], stay_alive=True, ) cluster.add_instance( "instance_24.5", with_zookeeper=True, image="clickhouse/clickhouse-server", tag="24.5", stay_alive=True, user_configs=[ "configs/users.xml", ], with_installed_binary=True, use_old_analyzer=True, ) logging.info("Starting cluster...") > cluster.start() test_storage_s3_queue/test.py:144: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:3119: in start self.wait_azurite_to_start() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = timeout = 180 def wait_azurite_to_start(self, timeout=180): from azure.storage.blob import BlobServiceClient connection_string = ( f"DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;" f"AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;" f"BlobEndpoint=http://127.0.0.1:{self.env_variables['AZURITE_PORT']}/devstoreaccount1;" ) time.sleep(1) start = time.time() while time.time() - start < timeout: try: blob_service_client = BlobServiceClient.from_connection_string( connection_string ) logging.debug(blob_service_client.get_account_information()) self.blob_service_client = blob_service_client return except Exception as ex: logging.debug("Can't connect to Azurite: %s", str(ex)) time.sleep(1) > raise Exception("Can't wait Azurite to start") E Exception: Can't wait Azurite to start helpers/cluster.py:2706: Exception ______________________ ERROR at setup of test_drop_table _______________________ [gw0] linux -- Python 3.10.12 /usr/bin/python3 @pytest.fixture(scope="module") def started_cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "instance", user_configs=["configs/users.xml"], with_minio=True, with_azurite=True, with_zookeeper=True, main_configs=[ "configs/zookeeper.xml", "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "instance2", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "old_instance", user_configs=["configs/users_old.xml"], with_zookeeper=True, # NOTE (vnemkov) Can't use altinity/clickhouse here since 23.8 doesn't hve S3Queue (and associated settings yet) image="clickhouse/clickhouse-server", tag="23.12", stay_alive=True, with_installed_binary=True, use_old_analyzer=True, ) cluster.add_instance( "instance_too_many_parts", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", "configs/merge_tree.xml", ], stay_alive=True, ) cluster.add_instance( "instance_24.5", with_zookeeper=True, image="clickhouse/clickhouse-server", tag="24.5", stay_alive=True, user_configs=[ "configs/users.xml", ], with_installed_binary=True, use_old_analyzer=True, ) logging.info("Starting cluster...") > cluster.start() test_storage_s3_queue/test.py:144: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:3119: in start self.wait_azurite_to_start() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = timeout = 180 def wait_azurite_to_start(self, timeout=180): from azure.storage.blob import BlobServiceClient connection_string = ( f"DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;" f"AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;" f"BlobEndpoint=http://127.0.0.1:{self.env_variables['AZURITE_PORT']}/devstoreaccount1;" ) time.sleep(1) start = time.time() while time.time() - start < timeout: try: blob_service_client = BlobServiceClient.from_connection_string( connection_string ) logging.debug(blob_service_client.get_account_information()) self.blob_service_client = blob_service_client return except Exception as ex: logging.debug("Can't connect to Azurite: %s", str(ex)) time.sleep(1) > raise Exception("Can't wait Azurite to start") E Exception: Can't wait Azurite to start helpers/cluster.py:2706: Exception ________________ ERROR at setup of test_exception_during_insert ________________ [gw0] linux -- Python 3.10.12 /usr/bin/python3 @pytest.fixture(scope="module") def started_cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "instance", user_configs=["configs/users.xml"], with_minio=True, with_azurite=True, with_zookeeper=True, main_configs=[ "configs/zookeeper.xml", "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "instance2", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "old_instance", user_configs=["configs/users_old.xml"], with_zookeeper=True, # NOTE (vnemkov) Can't use altinity/clickhouse here since 23.8 doesn't hve S3Queue (and associated settings yet) image="clickhouse/clickhouse-server", tag="23.12", stay_alive=True, with_installed_binary=True, use_old_analyzer=True, ) cluster.add_instance( "instance_too_many_parts", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", "configs/merge_tree.xml", ], stay_alive=True, ) cluster.add_instance( "instance_24.5", with_zookeeper=True, image="clickhouse/clickhouse-server", tag="24.5", stay_alive=True, user_configs=[ "configs/users.xml", ], with_installed_binary=True, use_old_analyzer=True, ) logging.info("Starting cluster...") > cluster.start() test_storage_s3_queue/test.py:144: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:3119: in start self.wait_azurite_to_start() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = timeout = 180 def wait_azurite_to_start(self, timeout=180): from azure.storage.blob import BlobServiceClient connection_string = ( f"DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;" f"AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;" f"BlobEndpoint=http://127.0.0.1:{self.env_variables['AZURITE_PORT']}/devstoreaccount1;" ) time.sleep(1) start = time.time() while time.time() - start < timeout: try: blob_service_client = BlobServiceClient.from_connection_string( connection_string ) logging.debug(blob_service_client.get_account_information()) self.blob_service_client = blob_service_client return except Exception as ex: logging.debug("Can't connect to Azurite: %s", str(ex)) time.sleep(1) > raise Exception("Can't wait Azurite to start") E Exception: Can't wait Azurite to start helpers/cluster.py:2706: Exception ___________ ERROR at setup of test_failed_retry[AzureQueue-ordered] ____________ [gw0] linux -- Python 3.10.12 /usr/bin/python3 @pytest.fixture(scope="module") def started_cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "instance", user_configs=["configs/users.xml"], with_minio=True, with_azurite=True, with_zookeeper=True, main_configs=[ "configs/zookeeper.xml", "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "instance2", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "old_instance", user_configs=["configs/users_old.xml"], with_zookeeper=True, # NOTE (vnemkov) Can't use altinity/clickhouse here since 23.8 doesn't hve S3Queue (and associated settings yet) image="clickhouse/clickhouse-server", tag="23.12", stay_alive=True, with_installed_binary=True, use_old_analyzer=True, ) cluster.add_instance( "instance_too_many_parts", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", "configs/merge_tree.xml", ], stay_alive=True, ) cluster.add_instance( "instance_24.5", with_zookeeper=True, image="clickhouse/clickhouse-server", tag="24.5", stay_alive=True, user_configs=[ "configs/users.xml", ], with_installed_binary=True, use_old_analyzer=True, ) logging.info("Starting cluster...") > cluster.start() test_storage_s3_queue/test.py:144: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:3119: in start self.wait_azurite_to_start() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = timeout = 180 def wait_azurite_to_start(self, timeout=180): from azure.storage.blob import BlobServiceClient connection_string = ( f"DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;" f"AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;" f"BlobEndpoint=http://127.0.0.1:{self.env_variables['AZURITE_PORT']}/devstoreaccount1;" ) time.sleep(1) start = time.time() while time.time() - start < timeout: try: blob_service_client = BlobServiceClient.from_connection_string( connection_string ) logging.debug(blob_service_client.get_account_information()) self.blob_service_client = blob_service_client return except Exception as ex: logging.debug("Can't connect to Azurite: %s", str(ex)) time.sleep(1) > raise Exception("Can't wait Azurite to start") E Exception: Can't wait Azurite to start helpers/cluster.py:2706: Exception __________ ERROR at setup of test_failed_retry[AzureQueue-unordered] ___________ [gw0] linux -- Python 3.10.12 /usr/bin/python3 @pytest.fixture(scope="module") def started_cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "instance", user_configs=["configs/users.xml"], with_minio=True, with_azurite=True, with_zookeeper=True, main_configs=[ "configs/zookeeper.xml", "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "instance2", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "old_instance", user_configs=["configs/users_old.xml"], with_zookeeper=True, # NOTE (vnemkov) Can't use altinity/clickhouse here since 23.8 doesn't hve S3Queue (and associated settings yet) image="clickhouse/clickhouse-server", tag="23.12", stay_alive=True, with_installed_binary=True, use_old_analyzer=True, ) cluster.add_instance( "instance_too_many_parts", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", "configs/merge_tree.xml", ], stay_alive=True, ) cluster.add_instance( "instance_24.5", with_zookeeper=True, image="clickhouse/clickhouse-server", tag="24.5", stay_alive=True, user_configs=[ "configs/users.xml", ], with_installed_binary=True, use_old_analyzer=True, ) logging.info("Starting cluster...") > cluster.start() test_storage_s3_queue/test.py:144: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:3119: in start self.wait_azurite_to_start() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = timeout = 180 def wait_azurite_to_start(self, timeout=180): from azure.storage.blob import BlobServiceClient connection_string = ( f"DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;" f"AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;" f"BlobEndpoint=http://127.0.0.1:{self.env_variables['AZURITE_PORT']}/devstoreaccount1;" ) time.sleep(1) start = time.time() while time.time() - start < timeout: try: blob_service_client = BlobServiceClient.from_connection_string( connection_string ) logging.debug(blob_service_client.get_account_information()) self.blob_service_client = blob_service_client return except Exception as ex: logging.debug("Can't connect to Azurite: %s", str(ex)) time.sleep(1) > raise Exception("Can't wait Azurite to start") E Exception: Can't wait Azurite to start helpers/cluster.py:2706: Exception _____________ ERROR at setup of test_failed_retry[S3Queue-ordered] _____________ [gw0] linux -- Python 3.10.12 /usr/bin/python3 @pytest.fixture(scope="module") def started_cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "instance", user_configs=["configs/users.xml"], with_minio=True, with_azurite=True, with_zookeeper=True, main_configs=[ "configs/zookeeper.xml", "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "instance2", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "old_instance", user_configs=["configs/users_old.xml"], with_zookeeper=True, # NOTE (vnemkov) Can't use altinity/clickhouse here since 23.8 doesn't hve S3Queue (and associated settings yet) image="clickhouse/clickhouse-server", tag="23.12", stay_alive=True, with_installed_binary=True, use_old_analyzer=True, ) cluster.add_instance( "instance_too_many_parts", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", "configs/merge_tree.xml", ], stay_alive=True, ) cluster.add_instance( "instance_24.5", with_zookeeper=True, image="clickhouse/clickhouse-server", tag="24.5", stay_alive=True, user_configs=[ "configs/users.xml", ], with_installed_binary=True, use_old_analyzer=True, ) logging.info("Starting cluster...") > cluster.start() test_storage_s3_queue/test.py:144: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:3119: in start self.wait_azurite_to_start() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = timeout = 180 def wait_azurite_to_start(self, timeout=180): from azure.storage.blob import BlobServiceClient connection_string = ( f"DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;" f"AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;" f"BlobEndpoint=http://127.0.0.1:{self.env_variables['AZURITE_PORT']}/devstoreaccount1;" ) time.sleep(1) start = time.time() while time.time() - start < timeout: try: blob_service_client = BlobServiceClient.from_connection_string( connection_string ) logging.debug(blob_service_client.get_account_information()) self.blob_service_client = blob_service_client return except Exception as ex: logging.debug("Can't connect to Azurite: %s", str(ex)) time.sleep(1) > raise Exception("Can't wait Azurite to start") E Exception: Can't wait Azurite to start helpers/cluster.py:2706: Exception ____________ ERROR at setup of test_failed_retry[S3Queue-unordered] ____________ [gw0] linux -- Python 3.10.12 /usr/bin/python3 @pytest.fixture(scope="module") def started_cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "instance", user_configs=["configs/users.xml"], with_minio=True, with_azurite=True, with_zookeeper=True, main_configs=[ "configs/zookeeper.xml", "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "instance2", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "old_instance", user_configs=["configs/users_old.xml"], with_zookeeper=True, # NOTE (vnemkov) Can't use altinity/clickhouse here since 23.8 doesn't hve S3Queue (and associated settings yet) image="clickhouse/clickhouse-server", tag="23.12", stay_alive=True, with_installed_binary=True, use_old_analyzer=True, ) cluster.add_instance( "instance_too_many_parts", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", "configs/merge_tree.xml", ], stay_alive=True, ) cluster.add_instance( "instance_24.5", with_zookeeper=True, image="clickhouse/clickhouse-server", tag="24.5", stay_alive=True, user_configs=[ "configs/users.xml", ], with_installed_binary=True, use_old_analyzer=True, ) logging.info("Starting cluster...") > cluster.start() test_storage_s3_queue/test.py:144: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:3119: in start self.wait_azurite_to_start() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = timeout = 180 def wait_azurite_to_start(self, timeout=180): from azure.storage.blob import BlobServiceClient connection_string = ( f"DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;" f"AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;" f"BlobEndpoint=http://127.0.0.1:{self.env_variables['AZURITE_PORT']}/devstoreaccount1;" ) time.sleep(1) start = time.time() while time.time() - start < timeout: try: blob_service_client = BlobServiceClient.from_connection_string( connection_string ) logging.debug(blob_service_client.get_account_information()) self.blob_service_client = blob_service_client return except Exception as ex: logging.debug("Can't connect to Azurite: %s", str(ex)) time.sleep(1) > raise Exception("Can't wait Azurite to start") E Exception: Can't wait Azurite to start helpers/cluster.py:2706: Exception ______________________ ERROR at setup of test_max_set_age ______________________ [gw0] linux -- Python 3.10.12 /usr/bin/python3 @pytest.fixture(scope="module") def started_cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "instance", user_configs=["configs/users.xml"], with_minio=True, with_azurite=True, with_zookeeper=True, main_configs=[ "configs/zookeeper.xml", "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "instance2", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "old_instance", user_configs=["configs/users_old.xml"], with_zookeeper=True, # NOTE (vnemkov) Can't use altinity/clickhouse here since 23.8 doesn't hve S3Queue (and associated settings yet) image="clickhouse/clickhouse-server", tag="23.12", stay_alive=True, with_installed_binary=True, use_old_analyzer=True, ) cluster.add_instance( "instance_too_many_parts", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", "configs/merge_tree.xml", ], stay_alive=True, ) cluster.add_instance( "instance_24.5", with_zookeeper=True, image="clickhouse/clickhouse-server", tag="24.5", stay_alive=True, user_configs=[ "configs/users.xml", ], with_installed_binary=True, use_old_analyzer=True, ) logging.info("Starting cluster...") > cluster.start() test_storage_s3_queue/test.py:144: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:3119: in start self.wait_azurite_to_start() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = timeout = 180 def wait_azurite_to_start(self, timeout=180): from azure.storage.blob import BlobServiceClient connection_string = ( f"DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;" f"AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;" f"BlobEndpoint=http://127.0.0.1:{self.env_variables['AZURITE_PORT']}/devstoreaccount1;" ) time.sleep(1) start = time.time() while time.time() - start < timeout: try: blob_service_client = BlobServiceClient.from_connection_string( connection_string ) logging.debug(blob_service_client.get_account_information()) self.blob_service_client = blob_service_client return except Exception as ex: logging.debug("Can't connect to Azurite: %s", str(ex)) time.sleep(1) > raise Exception("Can't wait Azurite to start") E Exception: Can't wait Azurite to start helpers/cluster.py:2706: Exception _____________________ ERROR at setup of test_max_set_size ______________________ [gw0] linux -- Python 3.10.12 /usr/bin/python3 @pytest.fixture(scope="module") def started_cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "instance", user_configs=["configs/users.xml"], with_minio=True, with_azurite=True, with_zookeeper=True, main_configs=[ "configs/zookeeper.xml", "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "instance2", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "old_instance", user_configs=["configs/users_old.xml"], with_zookeeper=True, # NOTE (vnemkov) Can't use altinity/clickhouse here since 23.8 doesn't hve S3Queue (and associated settings yet) image="clickhouse/clickhouse-server", tag="23.12", stay_alive=True, with_installed_binary=True, use_old_analyzer=True, ) cluster.add_instance( "instance_too_many_parts", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", "configs/merge_tree.xml", ], stay_alive=True, ) cluster.add_instance( "instance_24.5", with_zookeeper=True, image="clickhouse/clickhouse-server", tag="24.5", stay_alive=True, user_configs=[ "configs/users.xml", ], with_installed_binary=True, use_old_analyzer=True, ) logging.info("Starting cluster...") > cluster.start() test_storage_s3_queue/test.py:144: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:3119: in start self.wait_azurite_to_start() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = timeout = 180 def wait_azurite_to_start(self, timeout=180): from azure.storage.blob import BlobServiceClient connection_string = ( f"DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;" f"AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;" f"BlobEndpoint=http://127.0.0.1:{self.env_variables['AZURITE_PORT']}/devstoreaccount1;" ) time.sleep(1) start = time.time() while time.time() - start < timeout: try: blob_service_client = BlobServiceClient.from_connection_string( connection_string ) logging.debug(blob_service_client.get_account_information()) self.blob_service_client = blob_service_client return except Exception as ex: logging.debug("Can't connect to Azurite: %s", str(ex)) time.sleep(1) > raise Exception("Can't wait Azurite to start") E Exception: Can't wait Azurite to start helpers/cluster.py:2706: Exception _____________ ERROR at setup of test_multiple_tables_meta_mismatch _____________ [gw0] linux -- Python 3.10.12 /usr/bin/python3 @pytest.fixture(scope="module") def started_cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "instance", user_configs=["configs/users.xml"], with_minio=True, with_azurite=True, with_zookeeper=True, main_configs=[ "configs/zookeeper.xml", "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "instance2", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "old_instance", user_configs=["configs/users_old.xml"], with_zookeeper=True, # NOTE (vnemkov) Can't use altinity/clickhouse here since 23.8 doesn't hve S3Queue (and associated settings yet) image="clickhouse/clickhouse-server", tag="23.12", stay_alive=True, with_installed_binary=True, use_old_analyzer=True, ) cluster.add_instance( "instance_too_many_parts", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", "configs/merge_tree.xml", ], stay_alive=True, ) cluster.add_instance( "instance_24.5", with_zookeeper=True, image="clickhouse/clickhouse-server", tag="24.5", stay_alive=True, user_configs=[ "configs/users.xml", ], with_installed_binary=True, use_old_analyzer=True, ) logging.info("Starting cluster...") > cluster.start() test_storage_s3_queue/test.py:144: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:3119: in start self.wait_azurite_to_start() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = timeout = 180 def wait_azurite_to_start(self, timeout=180): from azure.storage.blob import BlobServiceClient connection_string = ( f"DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;" f"AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;" f"BlobEndpoint=http://127.0.0.1:{self.env_variables['AZURITE_PORT']}/devstoreaccount1;" ) time.sleep(1) start = time.time() while time.time() - start < timeout: try: blob_service_client = BlobServiceClient.from_connection_string( connection_string ) logging.debug(blob_service_client.get_account_information()) self.blob_service_client = blob_service_client return except Exception as ex: logging.debug("Can't connect to Azurite: %s", str(ex)) time.sleep(1) > raise Exception("Can't wait Azurite to start") E Exception: Can't wait Azurite to start helpers/cluster.py:2706: Exception _______ ERROR at setup of test_multiple_tables_streaming_sync[unordered] _______ [gw0] linux -- Python 3.10.12 /usr/bin/python3 @pytest.fixture(scope="module") def started_cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "instance", user_configs=["configs/users.xml"], with_minio=True, with_azurite=True, with_zookeeper=True, main_configs=[ "configs/zookeeper.xml", "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "instance2", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "old_instance", user_configs=["configs/users_old.xml"], with_zookeeper=True, # NOTE (vnemkov) Can't use altinity/clickhouse here since 23.8 doesn't hve S3Queue (and associated settings yet) image="clickhouse/clickhouse-server", tag="23.12", stay_alive=True, with_installed_binary=True, use_old_analyzer=True, ) cluster.add_instance( "instance_too_many_parts", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", "configs/merge_tree.xml", ], stay_alive=True, ) cluster.add_instance( "instance_24.5", with_zookeeper=True, image="clickhouse/clickhouse-server", tag="24.5", stay_alive=True, user_configs=[ "configs/users.xml", ], with_installed_binary=True, use_old_analyzer=True, ) logging.info("Starting cluster...") > cluster.start() test_storage_s3_queue/test.py:144: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:3119: in start self.wait_azurite_to_start() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = timeout = 180 def wait_azurite_to_start(self, timeout=180): from azure.storage.blob import BlobServiceClient connection_string = ( f"DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;" f"AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;" f"BlobEndpoint=http://127.0.0.1:{self.env_variables['AZURITE_PORT']}/devstoreaccount1;" ) time.sleep(1) start = time.time() while time.time() - start < timeout: try: blob_service_client = BlobServiceClient.from_connection_string( connection_string ) logging.debug(blob_service_client.get_account_information()) self.blob_service_client = blob_service_client return except Exception as ex: logging.debug("Can't connect to Azurite: %s", str(ex)) time.sleep(1) > raise Exception("Can't wait Azurite to start") E Exception: Can't wait Azurite to start helpers/cluster.py:2706: Exception __ ERROR at setup of test_multiple_tables_streaming_sync_distributed[ordered] __ [gw0] linux -- Python 3.10.12 /usr/bin/python3 @pytest.fixture(scope="module") def started_cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "instance", user_configs=["configs/users.xml"], with_minio=True, with_azurite=True, with_zookeeper=True, main_configs=[ "configs/zookeeper.xml", "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "instance2", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "old_instance", user_configs=["configs/users_old.xml"], with_zookeeper=True, # NOTE (vnemkov) Can't use altinity/clickhouse here since 23.8 doesn't hve S3Queue (and associated settings yet) image="clickhouse/clickhouse-server", tag="23.12", stay_alive=True, with_installed_binary=True, use_old_analyzer=True, ) cluster.add_instance( "instance_too_many_parts", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", "configs/merge_tree.xml", ], stay_alive=True, ) cluster.add_instance( "instance_24.5", with_zookeeper=True, image="clickhouse/clickhouse-server", tag="24.5", stay_alive=True, user_configs=[ "configs/users.xml", ], with_installed_binary=True, use_old_analyzer=True, ) logging.info("Starting cluster...") > cluster.start() test_storage_s3_queue/test.py:144: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:3119: in start self.wait_azurite_to_start() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = timeout = 180 def wait_azurite_to_start(self, timeout=180): from azure.storage.blob import BlobServiceClient connection_string = ( f"DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;" f"AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;" f"BlobEndpoint=http://127.0.0.1:{self.env_variables['AZURITE_PORT']}/devstoreaccount1;" ) time.sleep(1) start = time.time() while time.time() - start < timeout: try: blob_service_client = BlobServiceClient.from_connection_string( connection_string ) logging.debug(blob_service_client.get_account_information()) self.blob_service_client = blob_service_client return except Exception as ex: logging.debug("Can't connect to Azurite: %s", str(ex)) time.sleep(1) > raise Exception("Can't wait Azurite to start") E Exception: Can't wait Azurite to start helpers/cluster.py:2706: Exception _ ERROR at setup of test_multiple_tables_streaming_sync_distributed[unordered] _ [gw0] linux -- Python 3.10.12 /usr/bin/python3 @pytest.fixture(scope="module") def started_cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "instance", user_configs=["configs/users.xml"], with_minio=True, with_azurite=True, with_zookeeper=True, main_configs=[ "configs/zookeeper.xml", "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "instance2", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "old_instance", user_configs=["configs/users_old.xml"], with_zookeeper=True, # NOTE (vnemkov) Can't use altinity/clickhouse here since 23.8 doesn't hve S3Queue (and associated settings yet) image="clickhouse/clickhouse-server", tag="23.12", stay_alive=True, with_installed_binary=True, use_old_analyzer=True, ) cluster.add_instance( "instance_too_many_parts", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", "configs/merge_tree.xml", ], stay_alive=True, ) cluster.add_instance( "instance_24.5", with_zookeeper=True, image="clickhouse/clickhouse-server", tag="24.5", stay_alive=True, user_configs=[ "configs/users.xml", ], with_installed_binary=True, use_old_analyzer=True, ) logging.info("Starting cluster...") > cluster.start() test_storage_s3_queue/test.py:144: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:3119: in start self.wait_azurite_to_start() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = timeout = 180 def wait_azurite_to_start(self, timeout=180): from azure.storage.blob import BlobServiceClient connection_string = ( f"DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;" f"AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;" f"BlobEndpoint=http://127.0.0.1:{self.env_variables['AZURITE_PORT']}/devstoreaccount1;" ) time.sleep(1) start = time.time() while time.time() - start < timeout: try: blob_service_client = BlobServiceClient.from_connection_string( connection_string ) logging.debug(blob_service_client.get_account_information()) self.blob_service_client = blob_service_client return except Exception as ex: logging.debug("Can't connect to Azurite: %s", str(ex)) time.sleep(1) > raise Exception("Can't wait Azurite to start") E Exception: Can't wait Azurite to start helpers/cluster.py:2706: Exception _______________ ERROR at setup of test_processed_file_setting[1] _______________ [gw0] linux -- Python 3.10.12 /usr/bin/python3 @pytest.fixture(scope="module") def started_cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "instance", user_configs=["configs/users.xml"], with_minio=True, with_azurite=True, with_zookeeper=True, main_configs=[ "configs/zookeeper.xml", "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "instance2", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "old_instance", user_configs=["configs/users_old.xml"], with_zookeeper=True, # NOTE (vnemkov) Can't use altinity/clickhouse here since 23.8 doesn't hve S3Queue (and associated settings yet) image="clickhouse/clickhouse-server", tag="23.12", stay_alive=True, with_installed_binary=True, use_old_analyzer=True, ) cluster.add_instance( "instance_too_many_parts", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", "configs/merge_tree.xml", ], stay_alive=True, ) cluster.add_instance( "instance_24.5", with_zookeeper=True, image="clickhouse/clickhouse-server", tag="24.5", stay_alive=True, user_configs=[ "configs/users.xml", ], with_installed_binary=True, use_old_analyzer=True, ) logging.info("Starting cluster...") > cluster.start() test_storage_s3_queue/test.py:144: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:3119: in start self.wait_azurite_to_start() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = timeout = 180 def wait_azurite_to_start(self, timeout=180): from azure.storage.blob import BlobServiceClient connection_string = ( f"DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;" f"AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;" f"BlobEndpoint=http://127.0.0.1:{self.env_variables['AZURITE_PORT']}/devstoreaccount1;" ) time.sleep(1) start = time.time() while time.time() - start < timeout: try: blob_service_client = BlobServiceClient.from_connection_string( connection_string ) logging.debug(blob_service_client.get_account_information()) self.blob_service_client = blob_service_client return except Exception as ex: logging.debug("Can't connect to Azurite: %s", str(ex)) time.sleep(1) > raise Exception("Can't wait Azurite to start") E Exception: Can't wait Azurite to start helpers/cluster.py:2706: Exception _______________ ERROR at setup of test_processed_file_setting[5] _______________ [gw0] linux -- Python 3.10.12 /usr/bin/python3 @pytest.fixture(scope="module") def started_cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "instance", user_configs=["configs/users.xml"], with_minio=True, with_azurite=True, with_zookeeper=True, main_configs=[ "configs/zookeeper.xml", "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "instance2", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "old_instance", user_configs=["configs/users_old.xml"], with_zookeeper=True, # NOTE (vnemkov) Can't use altinity/clickhouse here since 23.8 doesn't hve S3Queue (and associated settings yet) image="clickhouse/clickhouse-server", tag="23.12", stay_alive=True, with_installed_binary=True, use_old_analyzer=True, ) cluster.add_instance( "instance_too_many_parts", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", "configs/merge_tree.xml", ], stay_alive=True, ) cluster.add_instance( "instance_24.5", with_zookeeper=True, image="clickhouse/clickhouse-server", tag="24.5", stay_alive=True, user_configs=[ "configs/users.xml", ], with_installed_binary=True, use_old_analyzer=True, ) logging.info("Starting cluster...") > cluster.start() test_storage_s3_queue/test.py:144: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:3119: in start self.wait_azurite_to_start() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = timeout = 180 def wait_azurite_to_start(self, timeout=180): from azure.storage.blob import BlobServiceClient connection_string = ( f"DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;" f"AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;" f"BlobEndpoint=http://127.0.0.1:{self.env_variables['AZURITE_PORT']}/devstoreaccount1;" ) time.sleep(1) start = time.time() while time.time() - start < timeout: try: blob_service_client = BlobServiceClient.from_connection_string( connection_string ) logging.debug(blob_service_client.get_account_information()) self.blob_service_client = blob_service_client return except Exception as ex: logging.debug("Can't connect to Azurite: %s", str(ex)) time.sleep(1) > raise Exception("Can't wait Azurite to start") E Exception: Can't wait Azurite to start helpers/cluster.py:2706: Exception _________ ERROR at setup of test_processed_file_setting_distributed[1] _________ [gw0] linux -- Python 3.10.12 /usr/bin/python3 @pytest.fixture(scope="module") def started_cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "instance", user_configs=["configs/users.xml"], with_minio=True, with_azurite=True, with_zookeeper=True, main_configs=[ "configs/zookeeper.xml", "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "instance2", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "old_instance", user_configs=["configs/users_old.xml"], with_zookeeper=True, # NOTE (vnemkov) Can't use altinity/clickhouse here since 23.8 doesn't hve S3Queue (and associated settings yet) image="clickhouse/clickhouse-server", tag="23.12", stay_alive=True, with_installed_binary=True, use_old_analyzer=True, ) cluster.add_instance( "instance_too_many_parts", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", "configs/merge_tree.xml", ], stay_alive=True, ) cluster.add_instance( "instance_24.5", with_zookeeper=True, image="clickhouse/clickhouse-server", tag="24.5", stay_alive=True, user_configs=[ "configs/users.xml", ], with_installed_binary=True, use_old_analyzer=True, ) logging.info("Starting cluster...") > cluster.start() test_storage_s3_queue/test.py:144: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:3119: in start self.wait_azurite_to_start() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = timeout = 180 def wait_azurite_to_start(self, timeout=180): from azure.storage.blob import BlobServiceClient connection_string = ( f"DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;" f"AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;" f"BlobEndpoint=http://127.0.0.1:{self.env_variables['AZURITE_PORT']}/devstoreaccount1;" ) time.sleep(1) start = time.time() while time.time() - start < timeout: try: blob_service_client = BlobServiceClient.from_connection_string( connection_string ) logging.debug(blob_service_client.get_account_information()) self.blob_service_client = blob_service_client return except Exception as ex: logging.debug("Can't connect to Azurite: %s", str(ex)) time.sleep(1) > raise Exception("Can't wait Azurite to start") E Exception: Can't wait Azurite to start helpers/cluster.py:2706: Exception _________ ERROR at setup of test_processed_file_setting_distributed[5] _________ [gw0] linux -- Python 3.10.12 /usr/bin/python3 @pytest.fixture(scope="module") def started_cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "instance", user_configs=["configs/users.xml"], with_minio=True, with_azurite=True, with_zookeeper=True, main_configs=[ "configs/zookeeper.xml", "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "instance2", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "old_instance", user_configs=["configs/users_old.xml"], with_zookeeper=True, # NOTE (vnemkov) Can't use altinity/clickhouse here since 23.8 doesn't hve S3Queue (and associated settings yet) image="clickhouse/clickhouse-server", tag="23.12", stay_alive=True, with_installed_binary=True, use_old_analyzer=True, ) cluster.add_instance( "instance_too_many_parts", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", "configs/merge_tree.xml", ], stay_alive=True, ) cluster.add_instance( "instance_24.5", with_zookeeper=True, image="clickhouse/clickhouse-server", tag="24.5", stay_alive=True, user_configs=[ "configs/users.xml", ], with_installed_binary=True, use_old_analyzer=True, ) logging.info("Starting cluster...") > cluster.start() test_storage_s3_queue/test.py:144: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:3119: in start self.wait_azurite_to_start() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = timeout = 180 def wait_azurite_to_start(self, timeout=180): from azure.storage.blob import BlobServiceClient connection_string = ( f"DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;" f"AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;" f"BlobEndpoint=http://127.0.0.1:{self.env_variables['AZURITE_PORT']}/devstoreaccount1;" ) time.sleep(1) start = time.time() while time.time() - start < timeout: try: blob_service_client = BlobServiceClient.from_connection_string( connection_string ) logging.debug(blob_service_client.get_account_information()) self.blob_service_client = blob_service_client return except Exception as ex: logging.debug("Can't connect to Azurite: %s", str(ex)) time.sleep(1) > raise Exception("Can't wait Azurite to start") E Exception: Can't wait Azurite to start helpers/cluster.py:2706: Exception ______________ ERROR at setup of test_processing_threads[ordered] ______________ [gw0] linux -- Python 3.10.12 /usr/bin/python3 @pytest.fixture(scope="module") def started_cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "instance", user_configs=["configs/users.xml"], with_minio=True, with_azurite=True, with_zookeeper=True, main_configs=[ "configs/zookeeper.xml", "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "instance2", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "old_instance", user_configs=["configs/users_old.xml"], with_zookeeper=True, # NOTE (vnemkov) Can't use altinity/clickhouse here since 23.8 doesn't hve S3Queue (and associated settings yet) image="clickhouse/clickhouse-server", tag="23.12", stay_alive=True, with_installed_binary=True, use_old_analyzer=True, ) cluster.add_instance( "instance_too_many_parts", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", "configs/merge_tree.xml", ], stay_alive=True, ) cluster.add_instance( "instance_24.5", with_zookeeper=True, image="clickhouse/clickhouse-server", tag="24.5", stay_alive=True, user_configs=[ "configs/users.xml", ], with_installed_binary=True, use_old_analyzer=True, ) logging.info("Starting cluster...") > cluster.start() test_storage_s3_queue/test.py:144: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:3119: in start self.wait_azurite_to_start() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = timeout = 180 def wait_azurite_to_start(self, timeout=180): from azure.storage.blob import BlobServiceClient connection_string = ( f"DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;" f"AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;" f"BlobEndpoint=http://127.0.0.1:{self.env_variables['AZURITE_PORT']}/devstoreaccount1;" ) time.sleep(1) start = time.time() while time.time() - start < timeout: try: blob_service_client = BlobServiceClient.from_connection_string( connection_string ) logging.debug(blob_service_client.get_account_information()) self.blob_service_client = blob_service_client return except Exception as ex: logging.debug("Can't connect to Azurite: %s", str(ex)) time.sleep(1) > raise Exception("Can't wait Azurite to start") E Exception: Can't wait Azurite to start helpers/cluster.py:2706: Exception _____________ ERROR at setup of test_processing_threads[unordered] _____________ [gw0] linux -- Python 3.10.12 /usr/bin/python3 @pytest.fixture(scope="module") def started_cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "instance", user_configs=["configs/users.xml"], with_minio=True, with_azurite=True, with_zookeeper=True, main_configs=[ "configs/zookeeper.xml", "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "instance2", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "old_instance", user_configs=["configs/users_old.xml"], with_zookeeper=True, # NOTE (vnemkov) Can't use altinity/clickhouse here since 23.8 doesn't hve S3Queue (and associated settings yet) image="clickhouse/clickhouse-server", tag="23.12", stay_alive=True, with_installed_binary=True, use_old_analyzer=True, ) cluster.add_instance( "instance_too_many_parts", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", "configs/merge_tree.xml", ], stay_alive=True, ) cluster.add_instance( "instance_24.5", with_zookeeper=True, image="clickhouse/clickhouse-server", tag="24.5", stay_alive=True, user_configs=[ "configs/users.xml", ], with_installed_binary=True, use_old_analyzer=True, ) logging.info("Starting cluster...") > cluster.start() test_storage_s3_queue/test.py:144: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:3119: in start self.wait_azurite_to_start() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = timeout = 180 def wait_azurite_to_start(self, timeout=180): from azure.storage.blob import BlobServiceClient connection_string = ( f"DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;" f"AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;" f"BlobEndpoint=http://127.0.0.1:{self.env_variables['AZURITE_PORT']}/devstoreaccount1;" ) time.sleep(1) start = time.time() while time.time() - start < timeout: try: blob_service_client = BlobServiceClient.from_connection_string( connection_string ) logging.debug(blob_service_client.get_account_information()) self.blob_service_client = blob_service_client return except Exception as ex: logging.debug("Can't connect to Azurite: %s", str(ex)) time.sleep(1) > raise Exception("Can't wait Azurite to start") E Exception: Can't wait Azurite to start helpers/cluster.py:2706: Exception ___________________ ERROR at setup of test_s3_client_reused ____________________ [gw0] linux -- Python 3.10.12 /usr/bin/python3 @pytest.fixture(scope="module") def started_cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "instance", user_configs=["configs/users.xml"], with_minio=True, with_azurite=True, with_zookeeper=True, main_configs=[ "configs/zookeeper.xml", "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "instance2", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "old_instance", user_configs=["configs/users_old.xml"], with_zookeeper=True, # NOTE (vnemkov) Can't use altinity/clickhouse here since 23.8 doesn't hve S3Queue (and associated settings yet) image="clickhouse/clickhouse-server", tag="23.12", stay_alive=True, with_installed_binary=True, use_old_analyzer=True, ) cluster.add_instance( "instance_too_many_parts", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", "configs/merge_tree.xml", ], stay_alive=True, ) cluster.add_instance( "instance_24.5", with_zookeeper=True, image="clickhouse/clickhouse-server", tag="24.5", stay_alive=True, user_configs=[ "configs/users.xml", ], with_installed_binary=True, use_old_analyzer=True, ) logging.info("Starting cluster...") > cluster.start() test_storage_s3_queue/test.py:144: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:3119: in start self.wait_azurite_to_start() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = timeout = 180 def wait_azurite_to_start(self, timeout=180): from azure.storage.blob import BlobServiceClient connection_string = ( f"DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;" f"AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;" f"BlobEndpoint=http://127.0.0.1:{self.env_variables['AZURITE_PORT']}/devstoreaccount1;" ) time.sleep(1) start = time.time() while time.time() - start < timeout: try: blob_service_client = BlobServiceClient.from_connection_string( connection_string ) logging.debug(blob_service_client.get_account_information()) self.blob_service_client = blob_service_client return except Exception as ex: logging.debug("Can't connect to Azurite: %s", str(ex)) time.sleep(1) > raise Exception("Can't wait Azurite to start") E Exception: Can't wait Azurite to start helpers/cluster.py:2706: Exception ____________________ ERROR at setup of test_settings_check _____________________ [gw0] linux -- Python 3.10.12 /usr/bin/python3 @pytest.fixture(scope="module") def started_cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "instance", user_configs=["configs/users.xml"], with_minio=True, with_azurite=True, with_zookeeper=True, main_configs=[ "configs/zookeeper.xml", "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "instance2", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "old_instance", user_configs=["configs/users_old.xml"], with_zookeeper=True, # NOTE (vnemkov) Can't use altinity/clickhouse here since 23.8 doesn't hve S3Queue (and associated settings yet) image="clickhouse/clickhouse-server", tag="23.12", stay_alive=True, with_installed_binary=True, use_old_analyzer=True, ) cluster.add_instance( "instance_too_many_parts", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", "configs/merge_tree.xml", ], stay_alive=True, ) cluster.add_instance( "instance_24.5", with_zookeeper=True, image="clickhouse/clickhouse-server", tag="24.5", stay_alive=True, user_configs=[ "configs/users.xml", ], with_installed_binary=True, use_old_analyzer=True, ) logging.info("Starting cluster...") > cluster.start() test_storage_s3_queue/test.py:144: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:3119: in start self.wait_azurite_to_start() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = timeout = 180 def wait_azurite_to_start(self, timeout=180): from azure.storage.blob import BlobServiceClient connection_string = ( f"DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;" f"AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;" f"BlobEndpoint=http://127.0.0.1:{self.env_variables['AZURITE_PORT']}/devstoreaccount1;" ) time.sleep(1) start = time.time() while time.time() - start < timeout: try: blob_service_client = BlobServiceClient.from_connection_string( connection_string ) logging.debug(blob_service_client.get_account_information()) self.blob_service_client = blob_service_client return except Exception as ex: logging.debug("Can't connect to Azurite: %s", str(ex)) time.sleep(1) > raise Exception("Can't wait Azurite to start") E Exception: Can't wait Azurite to start helpers/cluster.py:2706: Exception ___________________ ERROR at setup of test_shards[ordered-1] ___________________ [gw0] linux -- Python 3.10.12 /usr/bin/python3 @pytest.fixture(scope="module") def started_cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "instance", user_configs=["configs/users.xml"], with_minio=True, with_azurite=True, with_zookeeper=True, main_configs=[ "configs/zookeeper.xml", "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "instance2", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "old_instance", user_configs=["configs/users_old.xml"], with_zookeeper=True, # NOTE (vnemkov) Can't use altinity/clickhouse here since 23.8 doesn't hve S3Queue (and associated settings yet) image="clickhouse/clickhouse-server", tag="23.12", stay_alive=True, with_installed_binary=True, use_old_analyzer=True, ) cluster.add_instance( "instance_too_many_parts", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", "configs/merge_tree.xml", ], stay_alive=True, ) cluster.add_instance( "instance_24.5", with_zookeeper=True, image="clickhouse/clickhouse-server", tag="24.5", stay_alive=True, user_configs=[ "configs/users.xml", ], with_installed_binary=True, use_old_analyzer=True, ) logging.info("Starting cluster...") > cluster.start() test_storage_s3_queue/test.py:144: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:3119: in start self.wait_azurite_to_start() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = timeout = 180 def wait_azurite_to_start(self, timeout=180): from azure.storage.blob import BlobServiceClient connection_string = ( f"DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;" f"AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;" f"BlobEndpoint=http://127.0.0.1:{self.env_variables['AZURITE_PORT']}/devstoreaccount1;" ) time.sleep(1) start = time.time() while time.time() - start < timeout: try: blob_service_client = BlobServiceClient.from_connection_string( connection_string ) logging.debug(blob_service_client.get_account_information()) self.blob_service_client = blob_service_client return except Exception as ex: logging.debug("Can't connect to Azurite: %s", str(ex)) time.sleep(1) > raise Exception("Can't wait Azurite to start") E Exception: Can't wait Azurite to start helpers/cluster.py:2706: Exception ___________________ ERROR at setup of test_shards[ordered-8] ___________________ [gw0] linux -- Python 3.10.12 /usr/bin/python3 @pytest.fixture(scope="module") def started_cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "instance", user_configs=["configs/users.xml"], with_minio=True, with_azurite=True, with_zookeeper=True, main_configs=[ "configs/zookeeper.xml", "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "instance2", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "old_instance", user_configs=["configs/users_old.xml"], with_zookeeper=True, # NOTE (vnemkov) Can't use altinity/clickhouse here since 23.8 doesn't hve S3Queue (and associated settings yet) image="clickhouse/clickhouse-server", tag="23.12", stay_alive=True, with_installed_binary=True, use_old_analyzer=True, ) cluster.add_instance( "instance_too_many_parts", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", "configs/merge_tree.xml", ], stay_alive=True, ) cluster.add_instance( "instance_24.5", with_zookeeper=True, image="clickhouse/clickhouse-server", tag="24.5", stay_alive=True, user_configs=[ "configs/users.xml", ], with_installed_binary=True, use_old_analyzer=True, ) logging.info("Starting cluster...") > cluster.start() test_storage_s3_queue/test.py:144: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:3119: in start self.wait_azurite_to_start() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = timeout = 180 def wait_azurite_to_start(self, timeout=180): from azure.storage.blob import BlobServiceClient connection_string = ( f"DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;" f"AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;" f"BlobEndpoint=http://127.0.0.1:{self.env_variables['AZURITE_PORT']}/devstoreaccount1;" ) time.sleep(1) start = time.time() while time.time() - start < timeout: try: blob_service_client = BlobServiceClient.from_connection_string( connection_string ) logging.debug(blob_service_client.get_account_information()) self.blob_service_client = blob_service_client return except Exception as ex: logging.debug("Can't connect to Azurite: %s", str(ex)) time.sleep(1) > raise Exception("Can't wait Azurite to start") E Exception: Can't wait Azurite to start helpers/cluster.py:2706: Exception __________________ ERROR at setup of test_shards[unordered-1] __________________ [gw0] linux -- Python 3.10.12 /usr/bin/python3 @pytest.fixture(scope="module") def started_cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "instance", user_configs=["configs/users.xml"], with_minio=True, with_azurite=True, with_zookeeper=True, main_configs=[ "configs/zookeeper.xml", "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "instance2", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "old_instance", user_configs=["configs/users_old.xml"], with_zookeeper=True, # NOTE (vnemkov) Can't use altinity/clickhouse here since 23.8 doesn't hve S3Queue (and associated settings yet) image="clickhouse/clickhouse-server", tag="23.12", stay_alive=True, with_installed_binary=True, use_old_analyzer=True, ) cluster.add_instance( "instance_too_many_parts", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", "configs/merge_tree.xml", ], stay_alive=True, ) cluster.add_instance( "instance_24.5", with_zookeeper=True, image="clickhouse/clickhouse-server", tag="24.5", stay_alive=True, user_configs=[ "configs/users.xml", ], with_installed_binary=True, use_old_analyzer=True, ) logging.info("Starting cluster...") > cluster.start() test_storage_s3_queue/test.py:144: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:3119: in start self.wait_azurite_to_start() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = timeout = 180 def wait_azurite_to_start(self, timeout=180): from azure.storage.blob import BlobServiceClient connection_string = ( f"DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;" f"AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;" f"BlobEndpoint=http://127.0.0.1:{self.env_variables['AZURITE_PORT']}/devstoreaccount1;" ) time.sleep(1) start = time.time() while time.time() - start < timeout: try: blob_service_client = BlobServiceClient.from_connection_string( connection_string ) logging.debug(blob_service_client.get_account_information()) self.blob_service_client = blob_service_client return except Exception as ex: logging.debug("Can't connect to Azurite: %s", str(ex)) time.sleep(1) > raise Exception("Can't wait Azurite to start") E Exception: Can't wait Azurite to start helpers/cluster.py:2706: Exception __________________ ERROR at setup of test_shards[unordered-8] __________________ [gw0] linux -- Python 3.10.12 /usr/bin/python3 @pytest.fixture(scope="module") def started_cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "instance", user_configs=["configs/users.xml"], with_minio=True, with_azurite=True, with_zookeeper=True, main_configs=[ "configs/zookeeper.xml", "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "instance2", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "old_instance", user_configs=["configs/users_old.xml"], with_zookeeper=True, # NOTE (vnemkov) Can't use altinity/clickhouse here since 23.8 doesn't hve S3Queue (and associated settings yet) image="clickhouse/clickhouse-server", tag="23.12", stay_alive=True, with_installed_binary=True, use_old_analyzer=True, ) cluster.add_instance( "instance_too_many_parts", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", "configs/merge_tree.xml", ], stay_alive=True, ) cluster.add_instance( "instance_24.5", with_zookeeper=True, image="clickhouse/clickhouse-server", tag="24.5", stay_alive=True, user_configs=[ "configs/users.xml", ], with_installed_binary=True, use_old_analyzer=True, ) logging.info("Starting cluster...") > cluster.start() test_storage_s3_queue/test.py:144: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:3119: in start self.wait_azurite_to_start() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = timeout = 180 def wait_azurite_to_start(self, timeout=180): from azure.storage.blob import BlobServiceClient connection_string = ( f"DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;" f"AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;" f"BlobEndpoint=http://127.0.0.1:{self.env_variables['AZURITE_PORT']}/devstoreaccount1;" ) time.sleep(1) start = time.time() while time.time() - start < timeout: try: blob_service_client = BlobServiceClient.from_connection_string( connection_string ) logging.debug(blob_service_client.get_account_information()) self.blob_service_client = blob_service_client return except Exception as ex: logging.debug("Can't connect to Azurite: %s", str(ex)) time.sleep(1) > raise Exception("Can't wait Azurite to start") E Exception: Can't wait Azurite to start helpers/cluster.py:2706: Exception _____________ ERROR at setup of test_shards_distributed[ordered-1] _____________ [gw0] linux -- Python 3.10.12 /usr/bin/python3 @pytest.fixture(scope="module") def started_cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "instance", user_configs=["configs/users.xml"], with_minio=True, with_azurite=True, with_zookeeper=True, main_configs=[ "configs/zookeeper.xml", "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "instance2", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "old_instance", user_configs=["configs/users_old.xml"], with_zookeeper=True, # NOTE (vnemkov) Can't use altinity/clickhouse here since 23.8 doesn't hve S3Queue (and associated settings yet) image="clickhouse/clickhouse-server", tag="23.12", stay_alive=True, with_installed_binary=True, use_old_analyzer=True, ) cluster.add_instance( "instance_too_many_parts", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", "configs/merge_tree.xml", ], stay_alive=True, ) cluster.add_instance( "instance_24.5", with_zookeeper=True, image="clickhouse/clickhouse-server", tag="24.5", stay_alive=True, user_configs=[ "configs/users.xml", ], with_installed_binary=True, use_old_analyzer=True, ) logging.info("Starting cluster...") > cluster.start() test_storage_s3_queue/test.py:144: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:3119: in start self.wait_azurite_to_start() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = timeout = 180 def wait_azurite_to_start(self, timeout=180): from azure.storage.blob import BlobServiceClient connection_string = ( f"DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;" f"AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;" f"BlobEndpoint=http://127.0.0.1:{self.env_variables['AZURITE_PORT']}/devstoreaccount1;" ) time.sleep(1) start = time.time() while time.time() - start < timeout: try: blob_service_client = BlobServiceClient.from_connection_string( connection_string ) logging.debug(blob_service_client.get_account_information()) self.blob_service_client = blob_service_client return except Exception as ex: logging.debug("Can't connect to Azurite: %s", str(ex)) time.sleep(1) > raise Exception("Can't wait Azurite to start") E Exception: Can't wait Azurite to start helpers/cluster.py:2706: Exception _____________ ERROR at setup of test_shards_distributed[ordered-8] _____________ [gw0] linux -- Python 3.10.12 /usr/bin/python3 @pytest.fixture(scope="module") def started_cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "instance", user_configs=["configs/users.xml"], with_minio=True, with_azurite=True, with_zookeeper=True, main_configs=[ "configs/zookeeper.xml", "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "instance2", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "old_instance", user_configs=["configs/users_old.xml"], with_zookeeper=True, # NOTE (vnemkov) Can't use altinity/clickhouse here since 23.8 doesn't hve S3Queue (and associated settings yet) image="clickhouse/clickhouse-server", tag="23.12", stay_alive=True, with_installed_binary=True, use_old_analyzer=True, ) cluster.add_instance( "instance_too_many_parts", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", "configs/merge_tree.xml", ], stay_alive=True, ) cluster.add_instance( "instance_24.5", with_zookeeper=True, image="clickhouse/clickhouse-server", tag="24.5", stay_alive=True, user_configs=[ "configs/users.xml", ], with_installed_binary=True, use_old_analyzer=True, ) logging.info("Starting cluster...") > cluster.start() test_storage_s3_queue/test.py:144: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:3119: in start self.wait_azurite_to_start() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = timeout = 180 def wait_azurite_to_start(self, timeout=180): from azure.storage.blob import BlobServiceClient connection_string = ( f"DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;" f"AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;" f"BlobEndpoint=http://127.0.0.1:{self.env_variables['AZURITE_PORT']}/devstoreaccount1;" ) time.sleep(1) start = time.time() while time.time() - start < timeout: try: blob_service_client = BlobServiceClient.from_connection_string( connection_string ) logging.debug(blob_service_client.get_account_information()) self.blob_service_client = blob_service_client return except Exception as ex: logging.debug("Can't connect to Azurite: %s", str(ex)) time.sleep(1) > raise Exception("Can't wait Azurite to start") E Exception: Can't wait Azurite to start helpers/cluster.py:2706: Exception ____________ ERROR at setup of test_shards_distributed[unordered-1] ____________ [gw0] linux -- Python 3.10.12 /usr/bin/python3 @pytest.fixture(scope="module") def started_cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "instance", user_configs=["configs/users.xml"], with_minio=True, with_azurite=True, with_zookeeper=True, main_configs=[ "configs/zookeeper.xml", "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "instance2", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "old_instance", user_configs=["configs/users_old.xml"], with_zookeeper=True, # NOTE (vnemkov) Can't use altinity/clickhouse here since 23.8 doesn't hve S3Queue (and associated settings yet) image="clickhouse/clickhouse-server", tag="23.12", stay_alive=True, with_installed_binary=True, use_old_analyzer=True, ) cluster.add_instance( "instance_too_many_parts", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", "configs/merge_tree.xml", ], stay_alive=True, ) cluster.add_instance( "instance_24.5", with_zookeeper=True, image="clickhouse/clickhouse-server", tag="24.5", stay_alive=True, user_configs=[ "configs/users.xml", ], with_installed_binary=True, use_old_analyzer=True, ) logging.info("Starting cluster...") > cluster.start() test_storage_s3_queue/test.py:144: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:3119: in start self.wait_azurite_to_start() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = timeout = 180 def wait_azurite_to_start(self, timeout=180): from azure.storage.blob import BlobServiceClient connection_string = ( f"DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;" f"AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;" f"BlobEndpoint=http://127.0.0.1:{self.env_variables['AZURITE_PORT']}/devstoreaccount1;" ) time.sleep(1) start = time.time() while time.time() - start < timeout: try: blob_service_client = BlobServiceClient.from_connection_string( connection_string ) logging.debug(blob_service_client.get_account_information()) self.blob_service_client = blob_service_client return except Exception as ex: logging.debug("Can't connect to Azurite: %s", str(ex)) time.sleep(1) > raise Exception("Can't wait Azurite to start") E Exception: Can't wait Azurite to start helpers/cluster.py:2706: Exception ____________ ERROR at setup of test_shards_distributed[unordered-8] ____________ [gw0] linux -- Python 3.10.12 /usr/bin/python3 @pytest.fixture(scope="module") def started_cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "instance", user_configs=["configs/users.xml"], with_minio=True, with_azurite=True, with_zookeeper=True, main_configs=[ "configs/zookeeper.xml", "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "instance2", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "old_instance", user_configs=["configs/users_old.xml"], with_zookeeper=True, # NOTE (vnemkov) Can't use altinity/clickhouse here since 23.8 doesn't hve S3Queue (and associated settings yet) image="clickhouse/clickhouse-server", tag="23.12", stay_alive=True, with_installed_binary=True, use_old_analyzer=True, ) cluster.add_instance( "instance_too_many_parts", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", "configs/merge_tree.xml", ], stay_alive=True, ) cluster.add_instance( "instance_24.5", with_zookeeper=True, image="clickhouse/clickhouse-server", tag="24.5", stay_alive=True, user_configs=[ "configs/users.xml", ], with_installed_binary=True, use_old_analyzer=True, ) logging.info("Starting cluster...") > cluster.start() test_storage_s3_queue/test.py:144: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:3119: in start self.wait_azurite_to_start() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = timeout = 180 def wait_azurite_to_start(self, timeout=180): from azure.storage.blob import BlobServiceClient connection_string = ( f"DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;" f"AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;" f"BlobEndpoint=http://127.0.0.1:{self.env_variables['AZURITE_PORT']}/devstoreaccount1;" ) time.sleep(1) start = time.time() while time.time() - start < timeout: try: blob_service_client = BlobServiceClient.from_connection_string( connection_string ) logging.debug(blob_service_client.get_account_information()) self.blob_service_client = blob_service_client return except Exception as ex: logging.debug("Can't connect to Azurite: %s", str(ex)) time.sleep(1) > raise Exception("Can't wait Azurite to start") E Exception: Can't wait Azurite to start helpers/cluster.py:2706: Exception ___________ ERROR at setup of test_streaming_to_many_views[ordered] ____________ [gw0] linux -- Python 3.10.12 /usr/bin/python3 @pytest.fixture(scope="module") def started_cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "instance", user_configs=["configs/users.xml"], with_minio=True, with_azurite=True, with_zookeeper=True, main_configs=[ "configs/zookeeper.xml", "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "instance2", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "old_instance", user_configs=["configs/users_old.xml"], with_zookeeper=True, # NOTE (vnemkov) Can't use altinity/clickhouse here since 23.8 doesn't hve S3Queue (and associated settings yet) image="clickhouse/clickhouse-server", tag="23.12", stay_alive=True, with_installed_binary=True, use_old_analyzer=True, ) cluster.add_instance( "instance_too_many_parts", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", "configs/merge_tree.xml", ], stay_alive=True, ) cluster.add_instance( "instance_24.5", with_zookeeper=True, image="clickhouse/clickhouse-server", tag="24.5", stay_alive=True, user_configs=[ "configs/users.xml", ], with_installed_binary=True, use_old_analyzer=True, ) logging.info("Starting cluster...") > cluster.start() test_storage_s3_queue/test.py:144: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:3119: in start self.wait_azurite_to_start() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = timeout = 180 def wait_azurite_to_start(self, timeout=180): from azure.storage.blob import BlobServiceClient connection_string = ( f"DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;" f"AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;" f"BlobEndpoint=http://127.0.0.1:{self.env_variables['AZURITE_PORT']}/devstoreaccount1;" ) time.sleep(1) start = time.time() while time.time() - start < timeout: try: blob_service_client = BlobServiceClient.from_connection_string( connection_string ) logging.debug(blob_service_client.get_account_information()) self.blob_service_client = blob_service_client return except Exception as ex: logging.debug("Can't connect to Azurite: %s", str(ex)) time.sleep(1) > raise Exception("Can't wait Azurite to start") E Exception: Can't wait Azurite to start helpers/cluster.py:2706: Exception __________ ERROR at setup of test_streaming_to_many_views[unordered] ___________ [gw0] linux -- Python 3.10.12 /usr/bin/python3 @pytest.fixture(scope="module") def started_cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "instance", user_configs=["configs/users.xml"], with_minio=True, with_azurite=True, with_zookeeper=True, main_configs=[ "configs/zookeeper.xml", "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "instance2", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "old_instance", user_configs=["configs/users_old.xml"], with_zookeeper=True, # NOTE (vnemkov) Can't use altinity/clickhouse here since 23.8 doesn't hve S3Queue (and associated settings yet) image="clickhouse/clickhouse-server", tag="23.12", stay_alive=True, with_installed_binary=True, use_old_analyzer=True, ) cluster.add_instance( "instance_too_many_parts", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", "configs/merge_tree.xml", ], stay_alive=True, ) cluster.add_instance( "instance_24.5", with_zookeeper=True, image="clickhouse/clickhouse-server", tag="24.5", stay_alive=True, user_configs=[ "configs/users.xml", ], with_installed_binary=True, use_old_analyzer=True, ) logging.info("Starting cluster...") > cluster.start() test_storage_s3_queue/test.py:144: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:3119: in start self.wait_azurite_to_start() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = timeout = 180 def wait_azurite_to_start(self, timeout=180): from azure.storage.blob import BlobServiceClient connection_string = ( f"DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;" f"AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;" f"BlobEndpoint=http://127.0.0.1:{self.env_variables['AZURITE_PORT']}/devstoreaccount1;" ) time.sleep(1) start = time.time() while time.time() - start < timeout: try: blob_service_client = BlobServiceClient.from_connection_string( connection_string ) logging.debug(blob_service_client.get_account_information()) self.blob_service_client = blob_service_client return except Exception as ex: logging.debug("Can't connect to Azurite: %s", str(ex)) time.sleep(1) > raise Exception("Can't wait Azurite to start") E Exception: Can't wait Azurite to start helpers/cluster.py:2706: Exception ______________ ERROR at setup of test_streaming_to_view_[ordered] ______________ [gw0] linux -- Python 3.10.12 /usr/bin/python3 @pytest.fixture(scope="module") def started_cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "instance", user_configs=["configs/users.xml"], with_minio=True, with_azurite=True, with_zookeeper=True, main_configs=[ "configs/zookeeper.xml", "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "instance2", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "old_instance", user_configs=["configs/users_old.xml"], with_zookeeper=True, # NOTE (vnemkov) Can't use altinity/clickhouse here since 23.8 doesn't hve S3Queue (and associated settings yet) image="clickhouse/clickhouse-server", tag="23.12", stay_alive=True, with_installed_binary=True, use_old_analyzer=True, ) cluster.add_instance( "instance_too_many_parts", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", "configs/merge_tree.xml", ], stay_alive=True, ) cluster.add_instance( "instance_24.5", with_zookeeper=True, image="clickhouse/clickhouse-server", tag="24.5", stay_alive=True, user_configs=[ "configs/users.xml", ], with_installed_binary=True, use_old_analyzer=True, ) logging.info("Starting cluster...") > cluster.start() test_storage_s3_queue/test.py:144: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:3119: in start self.wait_azurite_to_start() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = timeout = 180 def wait_azurite_to_start(self, timeout=180): from azure.storage.blob import BlobServiceClient connection_string = ( f"DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;" f"AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;" f"BlobEndpoint=http://127.0.0.1:{self.env_variables['AZURITE_PORT']}/devstoreaccount1;" ) time.sleep(1) start = time.time() while time.time() - start < timeout: try: blob_service_client = BlobServiceClient.from_connection_string( connection_string ) logging.debug(blob_service_client.get_account_information()) self.blob_service_client = blob_service_client return except Exception as ex: logging.debug("Can't connect to Azurite: %s", str(ex)) time.sleep(1) > raise Exception("Can't wait Azurite to start") E Exception: Can't wait Azurite to start helpers/cluster.py:2706: Exception _____________ ERROR at setup of test_streaming_to_view_[unordered] _____________ [gw0] linux -- Python 3.10.12 /usr/bin/python3 @pytest.fixture(scope="module") def started_cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "instance", user_configs=["configs/users.xml"], with_minio=True, with_azurite=True, with_zookeeper=True, main_configs=[ "configs/zookeeper.xml", "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "instance2", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "old_instance", user_configs=["configs/users_old.xml"], with_zookeeper=True, # NOTE (vnemkov) Can't use altinity/clickhouse here since 23.8 doesn't hve S3Queue (and associated settings yet) image="clickhouse/clickhouse-server", tag="23.12", stay_alive=True, with_installed_binary=True, use_old_analyzer=True, ) cluster.add_instance( "instance_too_many_parts", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", "configs/merge_tree.xml", ], stay_alive=True, ) cluster.add_instance( "instance_24.5", with_zookeeper=True, image="clickhouse/clickhouse-server", tag="24.5", stay_alive=True, user_configs=[ "configs/users.xml", ], with_installed_binary=True, use_old_analyzer=True, ) logging.info("Starting cluster...") > cluster.start() test_storage_s3_queue/test.py:144: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:3119: in start self.wait_azurite_to_start() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = timeout = 180 def wait_azurite_to_start(self, timeout=180): from azure.storage.blob import BlobServiceClient connection_string = ( f"DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;" f"AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;" f"BlobEndpoint=http://127.0.0.1:{self.env_variables['AZURITE_PORT']}/devstoreaccount1;" ) time.sleep(1) start = time.time() while time.time() - start < timeout: try: blob_service_client = BlobServiceClient.from_connection_string( connection_string ) logging.debug(blob_service_client.get_account_information()) self.blob_service_client = blob_service_client return except Exception as ex: logging.debug("Can't connect to Azurite: %s", str(ex)) time.sleep(1) > raise Exception("Can't wait Azurite to start") E Exception: Can't wait Azurite to start helpers/cluster.py:2706: Exception ________________________ ERROR at setup of test_upgrade ________________________ [gw0] linux -- Python 3.10.12 /usr/bin/python3 @pytest.fixture(scope="module") def started_cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "instance", user_configs=["configs/users.xml"], with_minio=True, with_azurite=True, with_zookeeper=True, main_configs=[ "configs/zookeeper.xml", "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "instance2", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "old_instance", user_configs=["configs/users_old.xml"], with_zookeeper=True, # NOTE (vnemkov) Can't use altinity/clickhouse here since 23.8 doesn't hve S3Queue (and associated settings yet) image="clickhouse/clickhouse-server", tag="23.12", stay_alive=True, with_installed_binary=True, use_old_analyzer=True, ) cluster.add_instance( "instance_too_many_parts", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", "configs/merge_tree.xml", ], stay_alive=True, ) cluster.add_instance( "instance_24.5", with_zookeeper=True, image="clickhouse/clickhouse-server", tag="24.5", stay_alive=True, user_configs=[ "configs/users.xml", ], with_installed_binary=True, use_old_analyzer=True, ) logging.info("Starting cluster...") > cluster.start() test_storage_s3_queue/test.py:144: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:3119: in start self.wait_azurite_to_start() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = timeout = 180 def wait_azurite_to_start(self, timeout=180): from azure.storage.blob import BlobServiceClient connection_string = ( f"DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;" f"AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;" f"BlobEndpoint=http://127.0.0.1:{self.env_variables['AZURITE_PORT']}/devstoreaccount1;" ) time.sleep(1) start = time.time() while time.time() - start < timeout: try: blob_service_client = BlobServiceClient.from_connection_string( connection_string ) logging.debug(blob_service_client.get_account_information()) self.blob_service_client = blob_service_client return except Exception as ex: logging.debug("Can't connect to Azurite: %s", str(ex)) time.sleep(1) > raise Exception("Can't wait Azurite to start") E Exception: Can't wait Azurite to start helpers/cluster.py:2706: Exception _______________________ ERROR at setup of test_upgrade_2 _______________________ [gw0] linux -- Python 3.10.12 /usr/bin/python3 @pytest.fixture(scope="module") def started_cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "instance", user_configs=["configs/users.xml"], with_minio=True, with_azurite=True, with_zookeeper=True, main_configs=[ "configs/zookeeper.xml", "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "instance2", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", ], stay_alive=True, ) cluster.add_instance( "old_instance", user_configs=["configs/users_old.xml"], with_zookeeper=True, # NOTE (vnemkov) Can't use altinity/clickhouse here since 23.8 doesn't hve S3Queue (and associated settings yet) image="clickhouse/clickhouse-server", tag="23.12", stay_alive=True, with_installed_binary=True, use_old_analyzer=True, ) cluster.add_instance( "instance_too_many_parts", user_configs=["configs/users.xml"], with_minio=True, with_zookeeper=True, main_configs=[ "configs/s3queue_log.xml", "configs/merge_tree.xml", ], stay_alive=True, ) cluster.add_instance( "instance_24.5", with_zookeeper=True, image="clickhouse/clickhouse-server", tag="24.5", stay_alive=True, user_configs=[ "configs/users.xml", ], with_installed_binary=True, use_old_analyzer=True, ) logging.info("Starting cluster...") > cluster.start() test_storage_s3_queue/test.py:144: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:3119: in start self.wait_azurite_to_start() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = timeout = 180 def wait_azurite_to_start(self, timeout=180): from azure.storage.blob import BlobServiceClient connection_string = ( f"DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;" f"AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;" f"BlobEndpoint=http://127.0.0.1:{self.env_variables['AZURITE_PORT']}/devstoreaccount1;" ) time.sleep(1) start = time.time() while time.time() - start < timeout: try: blob_service_client = BlobServiceClient.from_connection_string( connection_string ) logging.debug(blob_service_client.get_account_information()) self.blob_service_client = blob_service_client return except Exception as ex: logging.debug("Can't connect to Azurite: %s", str(ex)) time.sleep(1) > raise Exception("Can't wait Azurite to start") E Exception: Can't wait Azurite to start helpers/cluster.py:2706: Exception =============================== warnings summary =============================== test_throttling/test.py::test_backup_throttling[no_local_throttling] /usr/local/lib/python3.10/dist-packages/urllib3/connectionpool.py:1100: InsecureRequestWarning: Unverified HTTPS request is being made to host '172.16.6.5'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings warnings.warn( test_zookeeper_config/test_secure.py::test_secure_connection test_zookeeper_config/test_secure.py::test_secure_connection test_zookeeper_config/test_secure.py::test_secure_connection test_zookeeper_config/test_secure.py::test_secure_connection test_zookeeper_config/test_secure.py::test_secure_connection test_zookeeper_config/test_secure.py::test_secure_connection test_zookeeper_config/test_secure.py::test_secure_connection test_zookeeper_config/test_secure.py::test_secure_connection test_zookeeper_config/test_secure.py::test_secure_connection /usr/local/lib/python3.10/dist-packages/kazoo/handlers/utils.py:225: DeprecationWarning: ssl.PROTOCOL_TLS is deprecated context = ssl.SSLContext(ssl.PROTOCOL_SSLv23) -- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html ============================== slowest durations =============================== 325.07s setup test_storage_s3_queue/test.py::test_direct_select_file[unordered] 31.19s teardown test_throttling/test.py::test_write_throttling[user_remote_throttling] 29.94s call test_zookeeper_config/test_secure.py::test_secure_connection 26.97s teardown test_storage_url_with_proxy/test.py::test_s3_with_proxy_list 23.15s call test_undrop_query/test.py::test_undrop_drop_and_undrop_loop 21.33s call test_version_update_after_mutation/test.py::test_mutate_and_upgrade 20.48s setup test_zookeeper_config/test_secure.py::test_secure_connection 20.11s call test_throttling/test.py::test_backup_throttling_override 17.91s setup test_truncate_database/test_distributed.py::test_truncate_database_distributed 17.71s setup test_system_ddl_worker_queue/test.py::test_distributed_ddl_queue 17.66s teardown test_truncate_database/test_distributed.py::test_truncate_database_distributed 16.66s setup test_version_update_after_mutation/test.py::test_mutate_and_upgrade 14.54s call test_throttling/test.py::test_read_throttling[user_local_throttling] 13.75s call test_throttling/test.py::test_backup_throttling[user_remote_to_local_throttling] 13.20s call test_throttling/test.py::test_write_throttling[user_remote_throttling] 13.10s setup test_transactions/test.py::test_rollback_unfinished_on_restart1 12.66s setup test_storage_url_with_proxy/test.py::test_s3_with_proxy_list 12.57s setup test_throttling/test.py::test_backup_throttling[no_local_throttling] 12.51s teardown test_system_ddl_worker_queue/test.py::test_distributed_ddl_rubbish 12.26s call test_throttling/test.py::test_read_throttling[user_remote_throttling] 12.25s call test_throttling/test.py::test_backup_throttling[user_local_throttling] 12.09s call test_throttling/test.py::test_backup_throttling[user_local_to_remote_throttling] 11.89s teardown test_zookeeper_config/test_secure.py::test_secure_connection 11.72s call test_throttling/test.py::test_write_throttling[local_user_throttling] 11.29s call test_throttling/test.py::test_backup_throttling[server_local_to_remote_throttling] 11.13s call test_user_defined_object_persistence/test.py::test_persistence 10.68s teardown test_system_queries/test.py::test_system_flush_logs 9.81s call test_transactions/test.py::test_rollback_unfinished_on_restart2 9.28s call test_throttling/test.py::test_backup_throttling[server_remote_to_local_throttling] 9.24s call test_throttling/test.py::test_read_throttling[server_local_throttling] 9.23s teardown test_undrop_query/test.py::test_undrop_drop_and_undrop_loop 8.85s call test_throttling/test.py::test_write_throttling[local_server_throttling] 8.43s call test_throttling/test.py::test_backup_throttling[server_local_throttling] 8.41s call test_system_ddl_worker_queue/test.py::test_distributed_ddl_queue 8.18s call test_transactions/test.py::test_rollback_unfinished_on_restart1 8.01s teardown test_throttling/test.py::test_backup_throttling[no_local_to_remote_throttling] 7.89s setup test_system_queries/test.py::test_DROP_DNS_CACHE 7.86s call test_throttling/test.py::test_write_throttling[server_remote_throttling] 7.84s call test_system_ddl_worker_queue/test.py::test_distributed_ddl_rubbish 7.82s call test_throttling/test.py::test_read_throttling[server_remote_throttling] 7.54s teardown test_throttling/test.py::test_backup_throttling[no_remote_to_remote_throttling] 7.50s teardown test_throttling/test.py::test_backup_throttling[server_remote_to_local_throttling] 7.47s setup test_undrop_query/test.py::test_undrop_drop_and_undrop_loop 7.42s call test_throttling/test.py::test_max_mutations_bandwidth_for_server 7.41s call test_throttling/test.py::test_max_merges_bandwidth_for_server 7.05s teardown test_user_zero_database_access/test_user_zero_database_access.py::test_user_zero_database_access 7.04s call test_version_update_after_mutation/test.py::test_upgrade_while_mutation 6.93s setup test_user_zero_database_access/test_user_zero_database_access.py::test_user_zero_database_access 6.77s teardown test_transactions/test.py::test_rollback_unfinished_on_restart2 6.67s teardown test_throttling/test.py::test_backup_throttling[no_remote_to_local_throttling] 6.44s setup test_system_flush_logs/test.py::test_system_logs_exists 6.42s teardown test_throttling/test.py::test_backup_throttling[server_local_to_remote_throttling] 6.35s teardown test_version_update_after_mutation/test.py::test_upgrade_while_mutation 6.22s call test_throttling/test.py::test_backup_throttling[server_remote_to_remote_throttling] 6.21s teardown test_throttling/test.py::test_write_throttling[local_user_throttling] 6.11s teardown test_throttling/test.py::test_write_throttling[no_local_throttling] 6.09s teardown test_throttling/test.py::test_backup_throttling[user_local_to_remote_throttling] 6.09s teardown test_throttling/test.py::test_backup_throttling_override 6.08s teardown test_user_defined_object_persistence/test.py::test_persistence 5.98s teardown test_throttling/test.py::test_backup_throttling[server_local_throttling] 5.94s teardown test_throttling/test.py::test_read_throttling[no_local_throttling] 5.85s teardown test_throttling/test.py::test_read_throttling[user_remote_throttling] 5.84s teardown test_throttling/test.py::test_write_throttling[local_server_throttling] 5.83s teardown test_throttling/test.py::test_backup_throttling[user_local_throttling] 5.73s teardown test_throttling/test.py::test_backup_throttling[server_remote_to_remote_throttling] 5.68s teardown test_throttling/test.py::test_read_throttling[no_remote_throttling] 5.61s teardown test_system_flush_logs/test.py::test_system_suspend 5.47s teardown test_throttling/test.py::test_backup_throttling[no_local_throttling] 5.42s teardown test_throttling/test.py::test_max_merges_bandwidth_for_server 5.26s teardown test_throttling/test.py::test_read_throttling[server_local_throttling] 5.26s call test_throttling/test.py::test_backup_throttling[user_remote_to_remote_throttling] 5.09s call test_system_queries/test.py::test_system_flush_logs 5.05s teardown test_throttling/test.py::test_read_throttling[server_remote_throttling] 4.94s setup test_user_defined_object_persistence/test.py::test_persistence 4.91s teardown test_throttling/test.py::test_backup_throttling[user_remote_to_remote_throttling] 4.85s teardown test_throttling/test.py::test_write_throttling[server_remote_throttling] 4.78s teardown test_throttling/test.py::test_read_throttling[user_local_throttling] 4.75s teardown test_throttling/test.py::test_backup_throttling[user_remote_to_local_throttling] 4.60s teardown test_throttling/test.py::test_write_throttling[no_remote_throttling] 4.50s teardown test_throttling/test.py::test_max_mutations_bandwidth_for_server 3.09s call test_system_queries/test.py::test_DROP_DNS_CACHE 2.33s call test_system_flush_logs/test.py::test_system_logs_exists 2.26s call test_user_zero_database_access/test_user_zero_database_access.py::test_user_zero_database_access 2.06s call test_system_flush_logs/test.py::test_system_suspend 1.47s call test_system_queries/test.py::test_SYSTEM_RELOAD_DICTIONARY 1.21s call test_truncate_database/test_distributed.py::test_truncate_database_distributed 1.01s call test_throttling/test.py::test_backup_throttling[no_local_throttling] 0.92s call test_throttling/test.py::test_backup_throttling[no_remote_to_local_throttling] 0.90s call test_throttling/test.py::test_backup_throttling[no_remote_to_remote_throttling] 0.65s call test_throttling/test.py::test_backup_throttling[no_local_to_remote_throttling] 0.64s call test_storage_url_with_proxy/test.py::test_s3_with_proxy_list 0.54s call test_system_flush_logs/test.py::test_system_logs_non_empty_queue 0.46s call test_throttling/test.py::test_read_throttling[no_remote_throttling] 0.45s call test_system_queries/test.py::test_RELOAD_CONFIG_AND_MACROS 0.39s call test_throttling/test.py::test_write_throttling[no_remote_throttling] 0.35s call test_throttling/test.py::test_read_throttling[no_local_throttling] 0.29s call test_throttling/test.py::test_write_throttling[no_local_throttling] 0.01s setup test_system_ddl_worker_queue/test.py::test_distributed_ddl_rubbish 0.01s setup test_storage_s3_queue/test.py::test_direct_select_multiple_files[unordered] 0.01s setup test_throttling/test.py::test_backup_throttling[server_remote_to_remote_throttling] 0.01s teardown test_system_flush_logs/test.py::test_system_logs_non_empty_queue 0.01s teardown test_storage_s3_queue/test.py::test_upgrade_2 0.01s setup test_throttling/test.py::test_backup_throttling[user_remote_to_remote_throttling] 0.00s setup test_throttling/test.py::test_write_throttling[user_remote_throttling] 0.00s setup test_transactions/test.py::test_rollback_unfinished_on_restart2 0.00s teardown test_system_ddl_worker_queue/test.py::test_distributed_ddl_queue 0.00s setup test_system_queries/test.py::test_system_flush_logs 0.00s setup test_throttling/test.py::test_backup_throttling[no_remote_to_remote_throttling] 0.00s setup test_throttling/test.py::test_backup_throttling[server_local_throttling] 0.00s setup test_throttling/test.py::test_backup_throttling_override 0.00s setup test_throttling/test.py::test_backup_throttling[no_remote_to_local_throttling] 0.00s setup test_system_queries/test.py::test_SYSTEM_RELOAD_DICTIONARY 0.00s teardown test_version_update_after_mutation/test.py::test_mutate_and_upgrade 0.00s teardown test_storage_s3_queue/test.py::test_direct_select_file[unordered] 0.00s setup test_throttling/test.py::test_backup_throttling[no_local_to_remote_throttling] 0.00s setup test_system_flush_logs/test.py::test_system_logs_non_empty_queue 0.00s setup test_throttling/test.py::test_backup_throttling[user_local_to_remote_throttling] 0.00s setup test_system_queries/test.py::test_RELOAD_CONFIG_AND_MACROS 0.00s setup test_throttling/test.py::test_write_throttling[no_local_throttling] 0.00s setup test_throttling/test.py::test_read_throttling[no_remote_throttling] 0.00s setup test_throttling/test.py::test_read_throttling[user_local_throttling] 0.00s setup test_throttling/test.py::test_write_throttling[no_remote_throttling] 0.00s setup test_throttling/test.py::test_backup_throttling[server_remote_to_local_throttling] 0.00s setup test_throttling/test.py::test_write_throttling[local_server_throttling] 0.00s setup test_throttling/test.py::test_read_throttling[server_local_throttling] 0.00s setup test_throttling/test.py::test_write_throttling[server_remote_throttling] 0.00s setup test_throttling/test.py::test_read_throttling[server_remote_throttling] 0.00s teardown test_storage_s3_queue/test.py::test_streaming_to_view_[unordered] 0.00s setup test_throttling/test.py::test_backup_throttling[user_local_throttling] 0.00s setup test_throttling/test.py::test_read_throttling[user_remote_throttling] 0.00s setup test_throttling/test.py::test_backup_throttling[user_remote_to_local_throttling] 0.00s teardown test_system_queries/test.py::test_DROP_DNS_CACHE 0.00s setup test_throttling/test.py::test_write_throttling[local_user_throttling] 0.00s setup test_throttling/test.py::test_read_throttling[no_local_throttling] 0.00s teardown test_storage_s3_queue/test.py::test_shards_distributed[ordered-1] 0.00s teardown test_transactions/test.py::test_rollback_unfinished_on_restart1 0.00s setup test_throttling/test.py::test_max_merges_bandwidth_for_server 0.00s setup test_system_flush_logs/test.py::test_system_suspend 0.00s teardown test_system_flush_logs/test.py::test_system_logs_exists 0.00s setup test_throttling/test.py::test_backup_throttling[server_local_to_remote_throttling] 0.00s setup test_storage_s3_queue/test.py::test_drop_table 0.00s setup test_storage_s3_queue/test.py::test_upgrade 0.00s setup test_storage_s3_queue/test.py::test_failed_retry[S3Queue-ordered] 0.00s setup test_storage_s3_queue/test.py::test_direct_select_multiple_files[ordered] 0.00s setup test_throttling/test.py::test_max_mutations_bandwidth_for_server 0.00s setup test_storage_s3_queue/test.py::test_failed_retry[S3Queue-unordered] 0.00s setup test_storage_s3_queue/test.py::test_s3_client_reused 0.00s teardown test_storage_s3_queue/test.py::test_failed_retry[S3Queue-ordered] 0.00s setup test_storage_s3_queue/test.py::test_upgrade_2 0.00s teardown test_system_queries/test.py::test_SYSTEM_RELOAD_DICTIONARY 0.00s setup test_storage_s3_queue/test.py::test_processed_file_setting_distributed[5] 0.00s setup test_storage_s3_queue/test.py::test_shards[ordered-1] 0.00s teardown test_storage_s3_queue/test.py::test_processed_file_setting_distributed[1] 0.00s setup test_version_update_after_mutation/test.py::test_upgrade_while_mutation 0.00s setup test_storage_s3_queue/test.py::test_multiple_tables_meta_mismatch 0.00s setup test_storage_s3_queue/test.py::test_failed_retry[AzureQueue-unordered] 0.00s setup test_storage_s3_queue/test.py::test_streaming_to_many_views[unordered] 0.00s setup test_storage_s3_queue/test.py::test_shards[unordered-8] 0.00s teardown test_storage_s3_queue/test.py::test_direct_select_multiple_files[unordered] 0.00s setup test_storage_s3_queue/test.py::test_processing_threads[unordered] 0.00s setup test_storage_s3_queue/test.py::test_processed_file_setting[5] 0.00s teardown test_storage_s3_queue/test.py::test_shards[ordered-8] 0.00s setup test_storage_s3_queue/test.py::test_streaming_to_view_[unordered] 0.00s setup test_storage_s3_queue/test.py::test_failed_retry[AzureQueue-ordered] 0.00s setup test_storage_s3_queue/test.py::test_streaming_to_many_views[ordered] 0.00s setup test_storage_s3_queue/test.py::test_max_set_age 0.00s setup test_storage_s3_queue/test.py::test_streaming_to_view_[ordered] 0.00s setup test_storage_s3_queue/test.py::test_shards[unordered-1] 0.00s setup test_storage_s3_queue/test.py::test_exception_during_insert 0.00s setup test_storage_s3_queue/test.py::test_shards_distributed[unordered-8] 0.00s setup test_storage_s3_queue/test.py::test_shards_distributed[ordered-1] 0.00s setup test_storage_s3_queue/test.py::test_processed_file_setting_distributed[1] 0.00s setup test_storage_s3_queue/test.py::test_processed_file_setting[1] 0.00s setup test_storage_s3_queue/test.py::test_settings_check 0.00s teardown test_storage_s3_queue/test.py::test_shards[ordered-1] 0.00s setup test_storage_s3_queue/test.py::test_multiple_tables_streaming_sync_distributed[unordered] 0.00s setup test_storage_s3_queue/test.py::test_shards_distributed[unordered-1] 0.00s setup test_storage_s3_queue/test.py::test_max_set_size 0.00s setup test_storage_s3_queue/test.py::test_shards_distributed[ordered-8] 0.00s teardown test_storage_s3_queue/test.py::test_drop_table 0.00s setup test_storage_s3_queue/test.py::test_shards[ordered-8] 0.00s teardown test_storage_s3_queue/test.py::test_failed_retry[AzureQueue-ordered] 0.00s teardown test_storage_s3_queue/test.py::test_streaming_to_many_views[ordered] 0.00s teardown test_storage_s3_queue/test.py::test_processed_file_setting[5] 0.00s teardown test_storage_s3_queue/test.py::test_max_set_size 0.00s teardown test_storage_s3_queue/test.py::test_processing_threads[unordered] 0.00s setup test_storage_s3_queue/test.py::test_multiple_tables_streaming_sync_distributed[ordered] 0.00s teardown test_storage_s3_queue/test.py::test_streaming_to_view_[ordered] 0.00s teardown test_storage_s3_queue/test.py::test_failed_retry[S3Queue-unordered] 0.00s setup test_storage_s3_queue/test.py::test_processing_threads[ordered] 0.00s teardown test_storage_s3_queue/test.py::test_upgrade 0.00s setup test_storage_s3_queue/test.py::test_multiple_tables_streaming_sync[unordered] 0.00s teardown test_storage_s3_queue/test.py::test_max_set_age 0.00s teardown test_storage_s3_queue/test.py::test_processing_threads[ordered] 0.00s teardown test_storage_s3_queue/test.py::test_failed_retry[AzureQueue-unordered] 0.00s teardown test_storage_s3_queue/test.py::test_shards[unordered-1] 0.00s teardown test_storage_s3_queue/test.py::test_shards[unordered-8] 0.00s teardown test_storage_s3_queue/test.py::test_exception_during_insert 0.00s teardown test_storage_s3_queue/test.py::test_direct_select_multiple_files[ordered] 0.00s teardown test_storage_s3_queue/test.py::test_shards_distributed[unordered-1] 0.00s teardown test_storage_s3_queue/test.py::test_multiple_tables_meta_mismatch 0.00s teardown test_storage_s3_queue/test.py::test_multiple_tables_streaming_sync_distributed[ordered] 0.00s teardown test_storage_s3_queue/test.py::test_processed_file_setting[1] 0.00s teardown test_storage_s3_queue/test.py::test_s3_client_reused 0.00s teardown test_storage_s3_queue/test.py::test_shards_distributed[unordered-8] 0.00s teardown test_storage_s3_queue/test.py::test_processed_file_setting_distributed[5] 0.00s teardown test_storage_s3_queue/test.py::test_settings_check 0.00s teardown test_storage_s3_queue/test.py::test_streaming_to_many_views[unordered] 0.00s teardown test_storage_s3_queue/test.py::test_multiple_tables_streaming_sync[unordered] 0.00s teardown test_storage_s3_queue/test.py::test_shards_distributed[ordered-8] 0.00s teardown test_storage_s3_queue/test.py::test_multiple_tables_streaming_sync_distributed[unordered] 0.00s teardown test_system_queries/test.py::test_RELOAD_CONFIG_AND_MACROS =========================== short test summary info ============================ ERROR test_storage_s3_queue/test.py::test_direct_select_file[unordered] - Exc... ERROR test_storage_s3_queue/test.py::test_direct_select_multiple_files[ordered] ERROR test_storage_s3_queue/test.py::test_direct_select_multiple_files[unordered] ERROR test_storage_s3_queue/test.py::test_drop_table - Exception: Can't wait ... ERROR test_storage_s3_queue/test.py::test_exception_during_insert - Exception... ERROR test_storage_s3_queue/test.py::test_failed_retry[AzureQueue-ordered] - ... ERROR test_storage_s3_queue/test.py::test_failed_retry[AzureQueue-unordered] ERROR test_storage_s3_queue/test.py::test_failed_retry[S3Queue-ordered] - Exc... ERROR test_storage_s3_queue/test.py::test_failed_retry[S3Queue-unordered] - E... ERROR test_storage_s3_queue/test.py::test_max_set_age - Exception: Can't wait... ERROR test_storage_s3_queue/test.py::test_max_set_size - Exception: Can't wai... ERROR test_storage_s3_queue/test.py::test_multiple_tables_meta_mismatch - Exc... ERROR test_storage_s3_queue/test.py::test_multiple_tables_streaming_sync[unordered] ERROR test_storage_s3_queue/test.py::test_multiple_tables_streaming_sync_distributed[ordered] ERROR test_storage_s3_queue/test.py::test_multiple_tables_streaming_sync_distributed[unordered] ERROR test_storage_s3_queue/test.py::test_processed_file_setting[1] - Excepti... ERROR test_storage_s3_queue/test.py::test_processed_file_setting[5] - Excepti... ERROR test_storage_s3_queue/test.py::test_processed_file_setting_distributed[1] ERROR test_storage_s3_queue/test.py::test_processed_file_setting_distributed[5] ERROR test_storage_s3_queue/test.py::test_processing_threads[ordered] - Excep... ERROR test_storage_s3_queue/test.py::test_processing_threads[unordered] - Exc... ERROR test_storage_s3_queue/test.py::test_s3_client_reused - Exception: Can't... ERROR test_storage_s3_queue/test.py::test_settings_check - Exception: Can't w... ERROR test_storage_s3_queue/test.py::test_shards[ordered-1] - Exception: Can'... ERROR test_storage_s3_queue/test.py::test_shards[ordered-8] - Exception: Can'... ERROR test_storage_s3_queue/test.py::test_shards[unordered-1] - Exception: Ca... ERROR test_storage_s3_queue/test.py::test_shards[unordered-8] - Exception: Ca... ERROR test_storage_s3_queue/test.py::test_shards_distributed[ordered-1] - Exc... ERROR test_storage_s3_queue/test.py::test_shards_distributed[ordered-8] - Exc... ERROR test_storage_s3_queue/test.py::test_shards_distributed[unordered-1] - E... ERROR test_storage_s3_queue/test.py::test_shards_distributed[unordered-8] - E... ERROR test_storage_s3_queue/test.py::test_streaming_to_many_views[ordered] - ... ERROR test_storage_s3_queue/test.py::test_streaming_to_many_views[unordered] ERROR test_storage_s3_queue/test.py::test_streaming_to_view_[ordered] - Excep... ERROR test_storage_s3_queue/test.py::test_streaming_to_view_[unordered] - Exc... ERROR test_storage_s3_queue/test.py::test_upgrade - Exception: Can't wait Azu... ERROR test_storage_s3_queue/test.py::test_upgrade_2 - Exception: Can't wait A... PASSED test_system_flush_logs/test.py::test_system_logs_exists PASSED test_system_flush_logs/test.py::test_system_logs_non_empty_queue PASSED test_system_queries/test.py::test_DROP_DNS_CACHE PASSED test_system_flush_logs/test.py::test_system_suspend PASSED test_system_queries/test.py::test_RELOAD_CONFIG_AND_MACROS PASSED test_system_queries/test.py::test_SYSTEM_RELOAD_DICTIONARY PASSED test_storage_url_with_proxy/test.py::test_s3_with_proxy_list PASSED test_throttling/test.py::test_backup_throttling[no_local_throttling] PASSED test_system_queries/test.py::test_system_flush_logs PASSED test_truncate_database/test_distributed.py::test_truncate_database_distributed PASSED test_throttling/test.py::test_backup_throttling[no_local_to_remote_throttling] PASSED test_transactions/test.py::test_rollback_unfinished_on_restart1 PASSED test_system_ddl_worker_queue/test.py::test_distributed_ddl_queue PASSED test_throttling/test.py::test_backup_throttling[no_remote_to_local_throttling] PASSED test_undrop_query/test.py::test_undrop_drop_and_undrop_loop PASSED test_transactions/test.py::test_rollback_unfinished_on_restart2 PASSED test_system_ddl_worker_queue/test.py::test_distributed_ddl_rubbish PASSED test_throttling/test.py::test_backup_throttling[no_remote_to_remote_throttling] PASSED test_version_update_after_mutation/test.py::test_mutate_and_upgrade PASSED test_version_update_after_mutation/test.py::test_upgrade_while_mutation PASSED test_user_zero_database_access/test_user_zero_database_access.py::test_user_zero_database_access PASSED test_throttling/test.py::test_backup_throttling[server_local_throttling] PASSED test_user_defined_object_persistence/test.py::test_persistence PASSED test_throttling/test.py::test_backup_throttling[server_local_to_remote_throttling] PASSED test_throttling/test.py::test_backup_throttling[server_remote_to_local_throttling] PASSED test_throttling/test.py::test_backup_throttling[server_remote_to_remote_throttling] PASSED test_zookeeper_config/test_secure.py::test_secure_connection PASSED test_throttling/test.py::test_backup_throttling[user_local_throttling] PASSED test_throttling/test.py::test_backup_throttling[user_local_to_remote_throttling] PASSED test_throttling/test.py::test_backup_throttling[user_remote_to_local_throttling] PASSED test_throttling/test.py::test_backup_throttling[user_remote_to_remote_throttling] PASSED test_throttling/test.py::test_backup_throttling_override PASSED test_throttling/test.py::test_max_merges_bandwidth_for_server PASSED test_throttling/test.py::test_max_mutations_bandwidth_for_server PASSED test_throttling/test.py::test_read_throttling[no_local_throttling] PASSED test_throttling/test.py::test_read_throttling[no_remote_throttling] PASSED test_throttling/test.py::test_read_throttling[server_local_throttling] PASSED test_throttling/test.py::test_read_throttling[server_remote_throttling] PASSED test_throttling/test.py::test_read_throttling[user_local_throttling] PASSED test_throttling/test.py::test_read_throttling[user_remote_throttling] PASSED test_throttling/test.py::test_write_throttling[local_server_throttling] PASSED test_throttling/test.py::test_write_throttling[local_user_throttling] PASSED test_throttling/test.py::test_write_throttling[no_local_throttling] PASSED test_throttling/test.py::test_write_throttling[no_remote_throttling] PASSED test_throttling/test.py::test_write_throttling[server_remote_throttling] PASSED test_throttling/test.py::test_write_throttling[user_remote_throttling] ============ 46 passed, 10 warnings, 37 errors in 407.90s (0:06:47) ============ Traceback (most recent call last): File "/home/ubuntu/_work/ClickHouse/ClickHouse/tests/integration/./runner", line 445, in subprocess.check_call(cmd, shell=True) File "/usr/lib/python3.10/subprocess.py", line 369, in check_call raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command 'docker run --rm --name clickhouse_integration_tests_96vjt7 --privileged --dns-search='.' --volume=/home/ubuntu/_work/_temp/test/build/clickhouse-odbc-bridge:/clickhouse-odbc-bridge --volume=/home/ubuntu/_work/_temp/test/build/clickhouse:/clickhouse --volume=/home/ubuntu/_work/_temp/test/build/clickhouse-library-bridge:/clickhouse-library-bridge --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/programs/server:/clickhouse-config --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/tests/integration:/ClickHouse/tests/integration --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/utils/backupview:/ClickHouse/utils/backupview --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/utils/grpc-client/pb2:/ClickHouse/utils/grpc-client/pb2 --volume=/run:/run/host:ro --volume=clickhouse_integration_tests_volume:/var/lib/docker -e DOCKER_DOTNET_CLIENT_TAG=11de0b29a15d -e DOCKER_HELPER_TAG=2cffe1eae894 -e DOCKER_BASE_TAG=b74d853ebe76 -e DOCKER_KERBEROS_KDC_TAG=9391ecdee8d7 -e DOCKER_MYSQL_GOLANG_CLIENT_TAG=9bec2a638e6e -e DOCKER_MYSQL_JAVA_CLIENT_TAG=766bff31cfe4 -e DOCKER_MYSQL_JS_CLIENT_TAG=41ba7c2ec2a1 -e DOCKER_MYSQL_PHP_CLIENT_TAG=88be89c1e3b6 -e DOCKER_NGINX_DAV_TAG=b55ac9cd7519 -e DOCKER_POSTGRESQL_JAVA_CLIENT_TAG=a4eff5c7f4d6 -e DOCKER_PYTHON_BOTTLE_TAG=caad4729259e -e DOCKER_CLIENT_TIMEOUT=300 -e COMPOSE_HTTP_TIMEOUT=600 -e PYTHONUNBUFFERED=1 -e PYTEST_ADDOPTS="--dist=loadfile -n 10 -rfEps --run-id=0 --color=no --durations=0 'test_storage_s3_queue/test.py::test_direct_select_file[unordered]' 'test_storage_s3_queue/test.py::test_direct_select_multiple_files[ordered]' 'test_storage_s3_queue/test.py::test_direct_select_multiple_files[unordered]' test_storage_s3_queue/test.py::test_drop_table test_storage_s3_queue/test.py::test_exception_during_insert 'test_storage_s3_queue/test.py::test_failed_retry[AzureQueue-ordered]' 'test_storage_s3_queue/test.py::test_failed_retry[AzureQueue-unordered]' 'test_storage_s3_queue/test.py::test_failed_retry[S3Queue-ordered]' 'test_storage_s3_queue/test.py::test_failed_retry[S3Queue-unordered]' test_storage_s3_queue/test.py::test_max_set_age test_storage_s3_queue/test.py::test_max_set_size test_storage_s3_queue/test.py::test_multiple_tables_meta_mismatch 'test_storage_s3_queue/test.py::test_multiple_tables_streaming_sync[unordered]' 'test_storage_s3_queue/test.py::test_multiple_tables_streaming_sync_distributed[ordered]' 'test_storage_s3_queue/test.py::test_multiple_tables_streaming_sync_distributed[unordered]' 'test_storage_s3_queue/test.py::test_processed_file_setting[1]' 'test_storage_s3_queue/test.py::test_processed_file_setting[5]' 'test_storage_s3_queue/test.py::test_processed_file_setting_distributed[1]' 'test_storage_s3_queue/test.py::test_processed_file_setting_distributed[5]' 'test_storage_s3_queue/test.py::test_processing_threads[ordered]' 'test_storage_s3_queue/test.py::test_processing_threads[unordered]' test_storage_s3_queue/test.py::test_s3_client_reused test_storage_s3_queue/test.py::test_settings_check 'test_storage_s3_queue/test.py::test_shards[ordered-1]' 'test_storage_s3_queue/test.py::test_shards[ordered-8]' 'test_storage_s3_queue/test.py::test_shards[unordered-1]' 'test_storage_s3_queue/test.py::test_shards[unordered-8]' 'test_storage_s3_queue/test.py::test_shards_distributed[ordered-1]' 'test_storage_s3_queue/test.py::test_shards_distributed[ordered-8]' 'test_storage_s3_queue/test.py::test_shards_distributed[unordered-1]' 'test_storage_s3_queue/test.py::test_shards_distributed[unordered-8]' 'test_storage_s3_queue/test.py::test_streaming_to_many_views[ordered]' 'test_storage_s3_queue/test.py::test_streaming_to_many_views[unordered]' 'test_storage_s3_queue/test.py::test_streaming_to_view_[ordered]' 'test_storage_s3_queue/test.py::test_streaming_to_view_[unordered]' test_storage_s3_queue/test.py::test_upgrade test_storage_s3_queue/test.py::test_upgrade_2 test_storage_url_with_proxy/test.py::test_s3_with_proxy_list test_system_ddl_worker_queue/test.py::test_distributed_ddl_queue test_system_ddl_worker_queue/test.py::test_distributed_ddl_rubbish test_system_flush_logs/test.py::test_system_logs_exists test_system_flush_logs/test.py::test_system_logs_non_empty_queue test_system_flush_logs/test.py::test_system_suspend test_system_queries/test.py::test_DROP_DNS_CACHE test_system_queries/test.py::test_RELOAD_CONFIG_AND_MACROS test_system_queries/test.py::test_SYSTEM_RELOAD_DICTIONARY test_system_queries/test.py::test_system_flush_logs 'test_throttling/test.py::test_backup_throttling[no_local_throttling]' 'test_throttling/test.py::test_backup_throttling[no_local_to_remote_throttling]' 'test_throttling/test.py::test_backup_throttling[no_remote_to_local_throttling]' 'test_throttling/test.py::test_backup_throttling[no_remote_to_remote_throttling]' 'test_throttling/test.py::test_backup_throttling[server_local_throttling]' 'test_throttling/test.py::test_backup_throttling[server_local_to_remote_throttling]' 'test_throttling/test.py::test_backup_throttling[server_remote_to_local_throttling]' 'test_throttling/test.py::test_backup_throttling[server_remote_to_remote_throttling]' 'test_throttling/test.py::test_backup_throttling[user_local_throttling]' 'test_throttling/test.py::test_backup_throttling[user_local_to_remote_throttling]' 'test_throttling/test.py::test_backup_throttling[user_remote_to_local_throttling]' 'test_throttling/test.py::test_backup_throttling[user_remote_to_remote_throttling]' test_throttling/test.py::test_backup_throttling_override test_throttling/test.py::test_max_merges_bandwidth_for_server test_throttling/test.py::test_max_mutations_bandwidth_for_server 'test_throttling/test.py::test_read_throttling[no_local_throttling]' 'test_throttling/test.py::test_read_throttling[no_remote_throttling]' 'test_throttling/test.py::test_read_throttling[server_local_throttling]' 'test_throttling/test.py::test_read_throttling[server_remote_throttling]' 'test_throttling/test.py::test_read_throttling[user_local_throttling]' 'test_throttling/test.py::test_read_throttling[user_remote_throttling]' 'test_throttling/test.py::test_write_throttling[local_server_throttling]' 'test_throttling/test.py::test_write_throttling[local_user_throttling]' 'test_throttling/test.py::test_write_throttling[no_local_throttling]' 'test_throttling/test.py::test_write_throttling[no_remote_throttling]' 'test_throttling/test.py::test_write_throttling[server_remote_throttling]' 'test_throttling/test.py::test_write_throttling[user_remote_throttling]' test_transactions/test.py::test_rollback_unfinished_on_restart1 test_transactions/test.py::test_rollback_unfinished_on_restart2 test_truncate_database/test_distributed.py::test_truncate_database_distributed test_undrop_query/test.py::test_undrop_drop_and_undrop_loop test_user_defined_object_persistence/test.py::test_persistence test_user_zero_database_access/test_user_zero_database_access.py::test_user_zero_database_access test_version_update_after_mutation/test.py::test_mutate_and_upgrade test_version_update_after_mutation/test.py::test_upgrade_while_mutation test_zookeeper_config/test_secure.py::test_secure_connection -vvv" altinityinfra/integration-tests-runner:cf95077f4609 ' returned non-zero exit status 1.